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. Abstract 

(N 

' The aim of this paper is to demonstrate the feasibiUty of authenticated throughput-efficient 

, routing in an unrehable and dynamically changing synchronous network in which the majority 

of malicious insiders try to destroy and alter messages or disrupt communication in any way. 
, More specifically, in this paper we seek to answer the following question: Given a network in 

which the majority of nodes are controlled by a node-controlling adversary and whose topology 
, is changing every round, is it possible to develop a protocol with polynomially-bounded memory 

per processor that guarantees throughput-efficient and correct end-to-end communication? We 
answer the question affirmatively for extremely general corruption patterns: we only request 
that the topology of the network and the corruption pattern of the adversary leaves at least one 
path each round connecting the sender and receiver through honest nodes (though this path may 
change at every round). Out construction works in the public-key setting and enjoys bounded 
1^ . memory per processor (that does not depend on the amount of traffic and is polynomial in the 

' network size.) Our protocol achieves optimal transfer rate with negligible decoding error. We 

lO . stress that our protocol assumes no knowledge of which nodes are corrupted nor which path is 

reliable at any round, and is also fully distributed with nodes making decisions locally, so that 
they need not know the topology of the network at any time. 
op ' The optimality that we prove for our protocol is very strong. Given any routing protocol, we 

evaluate its efficiency (rate of message delivery) in the "worst case," that is with respect to the 
worst possible graph and against the worst possible (polynomially bounded) adversarial strategy 
(subject to the above mentioned connectivity constraints). Using this metric, we show that there 
does not exist any protocol that can be asymptotically superior (in terms of throughput) to ours 
. in this setting. 

^ We remark that the aim of our paper is to demonstrate via expHcit example the feasibility 

of throughput-efficient authenticated adversarial routing. However, we stress that out protocol 
is not intended to provide a practical solution, as due to its complexity, no attempt thus far has 
been made to make the protocol practical by reducing constants or the large (though polynomial) 
memory requirements per processor. 

Our result is related to recent work of Barak, Goldberg and Xiao in 2008 [8J who stud- 
ied fault locaHzation in networks assuming a private-key trusted setup setting. Our work, in 
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contrast, assumes a public-key PKI setup and aims at not only fault localization, but also trans- 
mission optimality. Among other things, our work answers one of the open questions posed 
in the Barak et. al. paper regarding fault localization on multiple paths. The use of a public- 
key setting to achieve strong error-correction results in networks was inspired by the work of 
MicaH, Peikert, Sudan and Wilson [13] who showed that classical error-correction against a 
polynomially-bounded adversary can be achieved with surprisingly high precision. Our work is 
also related to an interactive coding theorem of Rajagopalan and Schulman [14j who showed 
that in noisy-edge static-topology networks a constant overhead in communication can also be 
achieved (provided none of the processors are malicious) , thus establishing an optimal-rate rout- 
ing theorem for static-topology networks. Finally, our work is closely related and builds upon 
to the problem of End-To-End Communication in distributed networks, studied by Afek and 
Gafni pLj, Awebuch, Mansour, and Shavit and Afek, Awerbuch, Gafni, Mansour, Rosen, 
and Shavit [2], though none of these papers consider or ensure correctness in the setting of a 
node-controlling adversary that may corrupt the majority of the network. 

Keywords: Network Routing; Error-correction; Fault Localization; Multi-parity Computation 
in the presence of Dishonest Majority; Communication Complexity; End-to-End Communica- 
tion. 

1 Introduction 

Our goal is to design a routing protocol for an unreliable and dynamically changing synchronous 
network that is resilient against malicious insiders who may try to destroy and alter messages or 
disrupt communication in any way. We model the network as a communication graph G = (V, E) 
where each vertex is a processor and each edge is a communication link. We do not assume that the 
topology of this graph is fixed or known by the processors. Rather, we assume a complete graph on 
n vertices, where some of the edges are "up" and some are "down", and the status of each edge can 
change dynamically at any time. 

We concentrate on the most basic task, namely how two processors in the network can exchange 
information. Thus, we assume that there are two designated vertices, called the sender S and the 
receiver R, who wish to communicate with each other. The sender has an infinite read-once input 
tape of packets and the receiver has an infinite write-once output tape which is initially empty. We 
assume that packets are of some bounded size, and that any edge in the system that is "up" during 
some round can transmit only one packet (or control variables, also of bounded size) per round. 

We will evaluate our protocol using the following three considerations: 

1. Correctness. A protocol is correct if the sequence of packets output by the receiver is a 
prefix of packets appearing on the sender's input tape, without duplication or omission. 

2. Throughput. This measures the number of packets on the output tape as a function of the 
number of rounds that have passed. 

3. Processor Memory. This measures the memory required of each node by the protocol, 
independent of the number of packets to be transferred. 

All three considerations will be measured in the worst-case scenario as standards that are guar- 
anteed to exist regardless of adversarial interference. One can also evaluate a protocol based on 
its dependence on global information to make decisions. In the protocol we present in this paper, 
we will not assume there is any global view of the network available to the internal nodes. Such 
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protocols are termed "local control," in that each node can make all routing decisions based only 
the local conditions of its adjacent edges and neighbors. 

Our protocol is designed to be resilient against a malicious, polynomially-bounded adversary 
who may attempt to impact the correctness, throughput, and memory of our protocol by disrupting 
links between the nodes or taking direct control over the nodes and forcing them to deviate from our 
protocol in any manner the adversary wishes. In order to relate our work to previous results and 
to clarify the two main forms of adversarial interference, we describe two separate (yet coordinated 
with each other) adversarie^^: 

Edge-Scheduling Adversary. This adversary controls the links between nodes every round. 
More precisely, at each round, this adversary decides which edges in the network are up 
and which are down. We will say that the edge-scheduling adversary is conforming if for 
every round there is at least one path from the sender to the receiver (although the path 
may change each roundjl. The adversary can make any arbitrary poly-time computation to 
maximize interference in routing, so long as it remains conforming. 

Node-Controlling Adversary. This adversary controls the nodes of the network that it has 
corrupted. More precisely, each round this adversary decides which nodes to corrupt. Once 
corrupted, a node is forever under complete adversarial control and can behave in an arbitrary 
malicious manner. We say that the node-controlling adversary is conforming if every round 
there is a connection between the sender and receiver consisting of edges that are "up" for the 
round (as specified by the edge-scheduling adversary) and that passes through uncorrupted 
nodes. We emphasize that this path can change each round, and there is no other restriction 
on which nodes the node-controlling adversary may corrupt (allowing even a vast majority of 
corrupt nodes). 

There is another reason to view these adversaries as distinct: we deal with the challenges they 
pose to correctness, throughput, and memory in different ways. Namely, aside from the conforming 
condition, the edge-scheduling adversary cannot be controlled or eliminated. Edges themselves 
are not inherently "good" or "bad," so identifying an edge that has failed does not allow us to 
forever refuse the protocol to utilize this edge, as it may come back up at any time (and indeed 
it could form a crucial link on the path connecting the sender and receiver that the conforming 
assumption guarantees). In sum, we cannot hope to control or alter the behavior of the edge- 
scheduling adversary, but must come up with a protocol that works well regardless of the behavior 
of the ever-present (conforming) edge-scheduling adversary. 

By contrast, our protocol will limit the amount of influence the node-controlling adversary has 
on correctness, throughput, and memory. Speciflcally, we will show that if a node deviates from 
the protocol in a sufficiently destructive manner (in a well-defined sense), then our protocol will 
be able to identify it as corrupted in a timely fashion. Once a corrupt node has been identified, 
it will be eliminated from the network. Namely, our protocol will call for honest nodes to refuse 

^The separation into two separate adversaries is artificial: our protocol is secure whether edge-scheduling and 
corruption of nodes are performed by two separate adversaries that have different capabilities yet can coordinate 
their actions with each other, or this can be viewed as a single coordinated adversary. 

more general definition of an edge-scheduling adversary would be to allow completely arbitrary edge failures, 
with the exception that in the limit there is no permanent cut between the sender and receiver. However, this 
definition (while more general) greatly complicates the exposition, including the definition of throughput rate, and 
we do not treat it here. 
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all communication with nodes that have been identified as corrupcl. Thus, there is an inherent 
difference in how we handle the edge-scheduling adversary verses how we handle the node-controlling 
adversary. We can restrict the influence of the latter by eliminating the nodes it has corrupted, 
while the former must be dealt with in a more ever-lasting manner. 

1.1 Previous Work 

To motivate the importance of the problem we consider in this paper, and to emphasize the sig- 
nificance of our result, it will be useful to highlight recent works in related areas. To date, routing 
protocols that consider adversarial networks have been of two main flavors: End-to-End Com- 
munication protocols that consider dynamic topologies (a notion captured by our "edge-scheduling 
adversary"), and Fault Detection and Localization protocols, which handle devious behavior of nodes 
(as modeled by our "node-controlling adversary"). 

End-to-End Communication: One of the most relevant research directions to our paper is the 
notion of End-to-End communication in distributed networks, considered by Afek and Gafni [l], 
Awerbuch, Mansour and Shavit [7], Afek, Awebuch, Gafni, Mansour, Rosen, and Shavit [2], and 
Kushilevitz, Ostrovsky and Rosen [12] . Indeed, our starting point is the Slide protocoQ developed 
in these works. It was designed to perform end-to-end communication with bounded memory in a 
model where (using our terminology) an edge-scheduling adversary controls the edges (subject to the 
constraint that there is no permanent cut between the sender and receiver). The Slide protocol has 
proven to be incredibly useful in a variety of settings, including multi-commodity flow (Awerbuch 
and Leigthon |6j) and in developing routing protocols that compete well (in terms of packet loss) 
against an online bursty adversary ([!]). However, prior to our work there was no version of the 
Slide protocol that could handle malicious behavior of the nodes. A comparison of various versions 
of the Slide protocol and our protocol is featured in Figured] of Section [L2] below. 

Fault Detection and Localization Protocols: At the other end, there have been a number 
of works that explore the possibility of a node-controlling adversary that can corrupt nodes. In 
particular, there is a recent line of work that considers a network consisting of a single path from the 
sender to the receiver, culminating in the recent work of Barak, Goldberg and Xiao [8j (for further 
background on fault localization see references therein). In this model, the adversary can corrupt 
any node on the path (except the sender and receiver) in a dynamic and malicious manner. Since 
corrupting any node on the path will sever the honest connection between S and i2, the goal of a 
protocol in this model is not to guarantee that all messages sent to R are received. Instead, the 
goal is to detect faults when they occur and to localize the fault to a single edge. 

There have been many results that provide Fault Detection (FD) and Fault Localization (FL) 
in this model. In Barak et. al. [8], they formalize the definitions in this model and the notion of a 
secure FD/FL protocol, as well as providing lower bounds in terms of communication complexity to 
guarantee accurate fault detection/location in the presence of a node-controlling adversary. While 
the Barak et. al. paper has a similar flavor to our paper, we emphasize that their protocol does not 
seek to guarantee successful or efficient routing between the sender and receiver. Instead, their proof 
of security guarantees that if a packet is deleted, malicious nodes cannot collude to convince S that 

''The conforming assumption guarantees that the sender and receiver are incorruptible, and our protocol places 
the responsibility of identifying and eliminating corrupt nodes on these two nodes. 
""Also known in practical works as "gravitational flow" routing. 
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no fault occurred, nor can they persuade S into believing that the fault occurred on an honest edge. 
Localizing the fault in their paper relies on cryptographic tools, and in particular the assumption 
that one-way functions exist. Although utilizing these tools (such as MACs or Signature Schemes) 
increases communication cost, it is shown by Goldberg, Xiao, Barak, and Redford p!l] that the 
existence of a protocol that is able to securely detect faults (in the presence of a node-controlling 
adversary) implies the existence of one-way functions, and it is shown in Barak et. al. [8j that any 
protocol that is able to securely localize faults necessarily requires the intermediate nodes to have 
a trusted setup. The proofs of these results do not rely on the fact that there is a single path 
between S and R, and we can therefore extend them to the more general network encountered in 
our model to justify our use of cryptographic tools and a trusted setup assumption (i.e. PKI) to 
identify malicious behavior. 

Another paper that addresses routing in the Byzantine setting is the work of Awerbuch, Holmes, 
Nina- Rotary and Rubens [5], though this paper does not have a fully formal treatment of security, 
and indeed a counter-example that challenges its security is discussed in the appendix of [8]. 

Error-correction in the active setting: Due to space considerations, we will not be able 
to give a comprehensive account of all the work in this area. Instead we highlight some of the most 
relevant works and point out how they differ from our setting and results. For a lengthy treatment of 
error-correcting codes against polynomially bounded adversaries, we refer to the work of Micali at. 
al [l3] and references therein. It is important to note that this work deals with a graph with a single 
"noisy" edge, as modelled by an adversary who can partially control and modify information that 
crosses the edge. In particular, it does not address throughput efficiency or memory considerations 
in a full communication network, nor does it account for malicious behavior at the vertices. Also 
of relevance is the work on Rajagopalan and Schulman on error-correcting network coding [14] , 
where they show how to correct noisy edges during distributed computation. Their work does not 
consider actively malicious nodes, and thus is different from our setting. It should also be noted 
that their work utilizes Schulman's tree-codes [17] that allow length-flexible online error-correction. 
The important difference between our work and that of Schulman is that in our network setting, 
the amount of malicious activity of corrupt nodes is not restricted. 

1.2 Our Results 

To date, there has not been a protocol that has considered simultaneously a network susceptible 
to faults occurring due to edge-failures and faults occurring due to malicious activity of corrupt 
nodes. The end-to-end communication works are not secure when the nodes are allowed to become 
corrupted by a node-controlling adversary, and the fault detection and localization works focus on 
a single path for some duration of time, and do not consider a fully distributed routing protocol 
that utilizes the entire network and attempts to maximize throughput efficiency while guaranteeing 
correctness in the presence of edge-failures and corrupt nodes. Indeed, our work answers one of 
the open questions posed in the Barak et. al. paper regarding fault localization on multiple paths. 
In this paper we bridge the gap between these two research areas and obtain the first routing 
protocol simultaneously secure against both an edge-scheduling adversary and a node-controlling 
adversary, even if these two adversaries attack the network using an arbitrary coordinated poly- 
time strategy. Furthermore, our protocol achieves comparable efficiency standards in terms of 
throughput and processor memory as state-of-the-art protocols that are not secure against a node- 
controlling adversary and does so using local-control protocols. An informal statement of our result 
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and comparison of our protocol to existing protocols can be found below. Although not included 
in the table, we emphasize that the linear transmission rate that we achieve (assuming at least r? 
messages are sent) is asymptotically optimal, as any protocol operating in a network with a single 
path connecting sender and receiver can do no better than one packet per round. 

A ROUTING THEOREM FOR ADVERSARIAL NETWORKS (Informal): If one-way 
functions exist, then for any n-node graph and sufficiently large, there exists a trusted-setup //near 
throughput transmission protocol that can send messages in O(n^) rounds with 0(n^(A; + log n)) 
memory per processor that is resilient against any poly-time conforming Edge-Scheduling Adversary and 
any conforming poly-time Node-Controlling Adversary, with negligible (in k) probability of failure or 
decoding error. 





Secure Against 
Edge-Sched. Ad? 


Secure Against 
Node-Cntr. Ad? 


Processor 
Memory 


Throughput Rate 
X rounds^/(a;) packets 


Slide Protocol of [2] 


YES 


NO 


0(n^ logn) 


fix) = 0(x-n2) 


Slide Protocol of p] 


YES 


NO 


0(n log n) 


fix) = 0(x/n — n^) 


(folklore) 
(Flooding + Signatures) 


YES 


YES 


0(1) 


fix) = Oix/n — n^) 


(folklore) 
(Signatures + Sequence No.'s) 


YES 


YES 


unbounded 


fix) = 0(x — n^) 


Our Protocol 


YES 


YES 


0{n'^{k+\ogn)) 


fix) = Oix - n2) 



Figure 1: Comparison of Our Protocol to Related Existing Protocols and Folklore. 

2 Challenges and Naive Solutions 

Before proceeding, it will be useful to consider a couple of naive solutions that achieve the goal of 
correctness (but perform poorly in terms of throughput)^ and help to illustrate some of the technical 
challenges that our theorem resolves. Consider the approach of having the sender continuously flood 
a single signed packet into the network for n rounds. Since the conforming assumption guarantees 
that the network provides a path between the sender and receiver through honest nodes at every 
round, this packet will reach the receiver within n rounds, regardless of adversarial interference. 
After n rounds, the sender can begin flooding the network with the next packet, and so fortlH. 
Notice that this solution will require each processor to store and continuously broadcast a single 
packet at any time, and hence this solution achieves excellent efficiency in terms of processor memory. 
However, notice that the throughput rate is sub-linear, namely after x rounds, only Oix/n) packets 
have been outputted by the receiver. 

^An alternative approach would have the sender continue flooding the first packet, and upon receipt, the re- 
ceiver floods confirmation of receipt. This alternative solution requires sequence numbers to accompany pack- 
ets/confirmations, and the rule that internal nodes only keep and broadcast the packet and conflrmation with largest 
sequence number. Although this alternative may potentially speed things up, in the worst-case it will still take 0{n) 
rounds for a single packet/confirmation pair to be transmitted. 
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One idea to try to improve the throughput rate might be to have the sender streamhne the 
process, sending packets with ever-increasing sequence numbers without waiting for n rounds to 
pass (or signed acknowledgments from the receiver) before sending the next packet. In particular, 
across each of his edges the sender will send every packet once, waiting only for the neighboring 
node's confirmation of receipt before sending the next packet across that edge. The protocol calls for 
the internal nodes to act similarly. Analysis of this approach shows that not only has the attempt 
to improve throughput failed (it is still 0{x/n) in the worst-case scenario), but additionally this 
modification requires arbitrarily large (polynomial in n and k) processor memory, since achieving 
correctness in the dynamic topology of the graph will force the nodes to remember all of the packets 
they see until they have broadcasted them across all adjacent edges or seen confirmation of their 
receipt from the receiver. 

2.1 Challenges in Dealing with Node-Controlling Adversaries 

In this section, we discuss some potential strategies that the node-controlling and edge-scheduling 
adversaries may incorporate to disrupt network communication. Although our theorem will work in 
the presence of arbitrary malicious activity of the adversarial controlled nodes (except with negligible 
probability), it will be instructive to list a few obvious forms of devious behavior that our protocol 
must protect against. It is important to stress that this list is nof intended to be exhaustive. Indeed, 
we do not claim to know all the specific ways an arbitrary polynomially bounded adversary may 
force nodes to deviate from a given protocol, and in this paper we rigorously prove that our protocol 
is secure against all possible deviations. 

• Packet Deletion/Modification. Instead of forwarding a packet, a corrupt node "drops it to the 
floor" (i.e. deletes it or effectively deletes it by forever storing it in memory), or modifies the 
packet before passing it on. Another manifestation of this is if the sender /receiver requests 
fault localization information of the internal nodes, such as providing documentation of their 
interactions with neighbors. A corrupt node can then block or modify information that passes 
through it in attempt to hide malicious activity or implicate an honest node. 

• Introduction of Junk/Duplicate Packets. The adversary can attempt to disrupt communication 
flow and "jam" the network by having corrupted nodes introduce junk packets or re-broadcast 
old packets. Notice that junk packets can be handled by using cryptographic signatures to 
prevent introduction of "new" packets, but this does not control the re-transmission of old, 
correctly signed packets. 

• Disobedience of Transfer Rules. If the protocol specifies how nodes should make decisions on 
where to send packets, etc., then corrupt nodes can disregard these rules. This includes "lying" 
to adjacent nodes about their current state. 

• Coordination of Edge-Failures. The edge-scheduling adversary can attempt to disrupt commu- 
nication flow by scheduling edge-failures in any manner that is consistent with the conforming 
criterion. Coordinating edge failures can be used to impede correctness, memory, and through- 
put in various ways: e.g. packets may become lost across a failed edge, stuck at a suddenly 
isolated node, or arrive at the receiver out of order. A separate issue arises concerning fault 
localization: when the sender/receiver requests documentation from the internal nodes, the 
edge-scheduling adversary can slow progress of this information, as well as attempt to protect 

® We give a formal definition of the adversary in Section 13.21 
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corrupt nodes by allowing them to "play-dead" (setting all of its adjacent edges to be down), 
so that incriminating evidence cannot reach the sender. 

2.2 Highlights of Our Solution 

Our starting point is the Slide protocol [2], which has enjoyed practical success in networks with 
dynamic topologies, but is not secure against nodes that are allowed to behave maliciously. We 
provide a detailed description of our version of the Slide protocol in Section HJ but highlight the 
main ideas here. Begin by viewing the edges in the graph as consisting of two directed edges, and 
associate to each end of a directed edge a stack data-structure able to hold 2n packets and to be 
maintained by the node at that end. The protocol specifies the following simple, local condition for 
transferring a packet across a directed edge: if there are more packets in the stack at the originating 
end than the terminating end, transfer a packet across the edge. Similarly, within a node's local 
stacks, packets are shuffled to average out the stack heights along each of its edges. Intuitively, 
packet movement is analogous to the flow of water: high stacks create a pressure that force packets 
to "flow" to neighboring lower stacks. At the source, the sender maintains the pressure by filling his 
outgoing stacks (as long as there is room) while the receiver relieves pressure by consuming packets 
and keeping his stacks empty. Loosely speaking, packets traveling to nodes "near" the sender will 
therefore require a very large potential, packets traveling to nodes near the receiver will require 
a small potential, and packet transfers near intermediate nodes will require packages to have a 
moderate potential. Assuming these potential requirements exist, packets will pass from the sender 
with a high potential, and then "flow" downwards across nodes requiring less potential, all the way 
to the receiver. 

Because the Slide protocol provides a fully distributed protocol that works well against an edge- 
scheduling adversary, our starting point was to try to extend the protocol by using digital signature^ 
to provide resilience against Byzantine attacks and arbitrary malicious behavior of corrupt nodes. 
This proved to be a highly nontrivial task that required us to develop a lot of additional machinery, 
both in terms of additional protocol ideas and novel techniques for proving correctness. We give a 
detailed explanation of our techniques in Section [8] and formal pseudo-code in Section [H as well as 
providing rigorous proofs of security in Section [TOl However, below we first give a sample of some 
of the key ideas we used in ensuring our additional machinery would be provably secure against a 
node-controlling adversary, and yet not significantly affect throughput or memory, compared to the 
original Slide protocol: 

• Addressing the "Coordination of Edge-Scheduling" Issues. In the absence of a 
node-controlling adversary, previous versions of the Slide protocol (e.g. |[2j) are secure and 
efficient against an edge-scheduling adversary, and it will be useful to discuss how some of 
the challenges posed by a network with a dynamic topology are handled. First, note that the 
total capacity of the stack data-structure is bounded by 4n^. That is, each of the n nodes can 
hold at most 2n packets in each of their 2n stacks (along each directed edge) at any time. 

'^In this paper we use public-key operations to sign individual packets with control information. Clearly, this is too 
expensive to do per-packet in practice. There are methods of amortizing the cost of signatures by signing "batches" of 
packets; using private- key initialization [8llll|. or using a combination of private- key and public key operations, such 
as "on-line/off-line" signatures [9l I16|. For the sake of clarity and since the primary focus of our paper is theoretical 
feasibility, we restrict our attention to the straight-forward public-key setting without considering these additional 
cost-saving techniques. 
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— To handle the loss of packets due to an edge going down while transmitting a packet, 
a node is required to maintain a copy of each packet it transmits along an edge until it 
receives confirmation from the neighbor of successful receipt. 

— To handle packets becoming stuck in some internal node's stack due to edge failures, 
error- correction is utilized to allow the receiver to decode a full message without needing 
every packet. In particular, if an error-correcting code allowing a fraction of A faults is 
utilized, then since the capacity of the network is An^ packets, if the sender is able to 
pump 4n^/A codeword packets into the network and there is no malicious deletion or 
modification of packets, then the receiver will necessarily have received enough packets 
to decode the message. 

— The Slide protocol has a natural bound in terms of memory per processor of 0{n? logn) 
bits, where the bottleneck is the possibility of a node holding up to 2n? packets in its 
stacks, where each packet requires O(logn) bits to describe its position in the code. 

Of course, these techniques are only valid if nodes are acting honestly, which leads us to our 
first extension idea. 

• Handling Packet Modification and Introduction of Junk Packets. Before insert- 
ing any packets into the network, the sender will authenticate each packet using his digital 
signature, and intermediate nodes and the receiver never accept or forward messages not ap- 
propriately signed. This simultaneously prevents honest nodes becoming bogged down with 
junk packets, as well as ensuring that if the receiver has obtained enough authenticated packets 
to decode, a node-controlling adversary cannot impede the successful decoding of the mes- 
sage as the integrity of the codeword packets is guaranteed by the inforgibility of the sender's 
signature. 

• Fault Detection. In the absence of a node-controlling adversary, our protocol looks almost 
identical to the Slide protocol of [2], with the addition of signatures that accompany all 
interactions between two nodes. First, the sender attempts to pump the An^/X codeword 
packets of the first message into the network, with packet movement exactly as in the original 
Slide protocol. We consider all possible outcomes: 

1. The sender is able to insert all codeword packets and the receiver is able to decode. In this 
case, the message was transmitted successfully, and our protocol moves to transfer the 
next message. 

2. The sender is able to insert all codeword packets, but the receiver has not received enough to 
decode. In this case, the receiver floods the network with a single-bit message indicating 
packet deletion has occurred. 

3. The sender is able to insert all codeword packets, but the receiver cannot decode because he 
has received duplicated packets. Although the sender's authenticating signature guarantees 
the receiver will not receive junk or modified packets, a corrupt node is able to duplicate 
valid packets. Therefore, the receiver may receive enough packets to decode, but cannot 
because he has received duplicates. In this case, the receiver floods the network with a 
single message indicating the label of a duplicated packet. 

4. After some amount of time, the sender still has not inserted all codeword packets. In this 
case, the duplication of old packets is so severe that the network has become jammed, 
and the sender is prevented from inserting packets even along the honest path that 



9 



the conforming assumption guarantees. If the sender beheves the jamming cannot be 
accounted for bv edge-failures alone, he will halt transmission and move to localizing 
a corrupt nodq£|. One contribution this paper makes is to prove a lower bound on the 
insertion rate of the sender for the Slide protocol in the absence of the node- controlling 
adversary. This bound not only alerts the sender when the jamming he is experiencing 
exceeds what can be expected in the absence of corrupt nodes, but it also provides a 
mechanism for localizing the offending node(s). 

The above four cases exhaust all possibilities. Furthermore, if a transmission is not successful, 
the sender is not only able to detect the fact that malicious activity has occured, but he is also 
able to distinguish the form of the malicious activity, i.e. which case 2-4 he is in. Meanwhile, 
for the top case, our protocol enjoys (within a constant factor) an equivalent throughput rate 
as the original Slide protocol. 

• Fault Localization. Once a fault has been detected, it remains to describe how to localize 
the problem to the offending node. To this end, we use digital signatures to achieve a new 
mechanism we call "Routing with Responsibility." By forcing nodes to sign key parts of every 
communication with their neighbors during the transfer of packets, they can later be held 
accountable for their actions. In particular, once the sender has identified the reason for 
failure (cases 2-4 above), he will request all internal nodes to return status reports, which 
are signatures on the relevant parts of the communication with their neighbors. We then 
prove in each case that with the complete status report from every node, the sender can with 
overwhelming probability identify and eliminate a corrupt node. Of course, malicious nodes 
may choose not to send incriminating information. We handle this separately as explained 
below. 

• Processor Memory. The signatures on the communication a node has with its neighbors 
for the purpose of fault localization is a burden on the memory required of each processor 
that is not encountered in the original Slide protocol. One major challenge was to reduce 
the amount of signed information each node must maintain as much as possible, while still 
guaranteeing that each node has maintained "enough" information to identify a corrupt node 
in the case of arbitrary malicious activity leading to a failure of type 2-4 above. The content 
of Theorem 18.21 in Section [8] demonstrates that the extra memory required of our protocol is 
a factor of higher than that of the original Slide protocol. 

• Incomplete Information. As already mentioned, we show that regardless of the reason of 
failure 2-4 above, once the sender receives the status reports from every node, a corrupt node 
can be identified. However, this relies on the sender obtaining all of the relevant information; 
the absence of even a single node's information can prevent the localization of a fault. We 
address this challenge in the following ways: 

1. We minimize the amount of information the sender requires of each node. This way, a 
node need not be connected to the sender for very many rounds in order for the sender 

*We emphasize here the importance that the sender is able to distinguish the case that the jamming is a result of 
the edge-scheduling adversary's controlling of edges verses the case that a corrupt node is duplicating packets. After 
all, in the case of the former, there is no reward for "localizing" the fault to an edge that has failed, as all edges 
are controlled by the edge-scheduling adversary, and therefore no edge is inherently better than another. But in the 
case a node is duplicating packets, if the sender can identify the node, it can eliminate it and effectively reduce the 
node-controlling adversary's ability to disrupt communication in the future. 
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to receive its information. Specifically, regardless of the reason for failure 2-4 above, a 
status report consists of only n pieces of information from each node, i.e. one packet for 
each of its edges. 

2. If the sender does not have the n pieces of information from a node, it cannot afford to 
wait indefinitely. After all, the edge-scheduling adversary may keep the node disconnected 
indefinitely, or a corrupt node may simply refuse to respond. For this purpose, we create 
a blacklist for non-responding nodes, which will disallow them from transferring codeword 
packets in the future. This way, anytime the receiver fails to decode a codeword as in 
cases 2-4 above, the sender can request the information he needs, blacklist nodes not 
responding within some short amount of time, and then re-attempt to transmit the 
codeword using only non-blacklisted nodes. Nodes should not transfer codeword packets 
to blacklisted nodes, but they do still communicate with them to transfer the information 
the sender has requested. If a new transmission again fails, the sender will only need to 
request information from nodes that were participating, i.e. he will not need to collect 
new information from blacklisted nodes (although the nodes will remain blacklisted until 
the sender gets the original information he requested of them). Nodes will be removed 
from the blacklist and re-allowed to route codeword packets as soon as the sender receives 
their information. 

• The Blacklist. Blacklisting nodes is a delicate matter; we want to place malicious nodes 
"playing- dead" on this list, while at the same time we don't want honest nodes that are tem- 
porarily disconnected from being on this list for too long. We show in Theorem 18.11 and Lemma 
110.91 that the occasional honest node that gets put on the blacklist won't significantly hinder 
packet transmission. Intuitively, this is true because any honest node that is an important 
link between the sender and receiver will not remain on the blacklist for very long, as his 
connection to the sender guarantees the sender will receive all requested information from the 
node in a timely manner. 

Ultimately, the blacklist allows us to control the amount of malicious activity a single 
corrupt node can contribute to. Indeed, we show that each failed message transmission (cases 
2-4 above) can be localized (eventually) to (at least) one corrupt node. More precisely, the 
blacklist allows us to argue that malicious activity can cause at most n failed transmissions 
before a corrupt node can necessarily be identified and eliminated. Since there are at most n 
corrupt nodes, this bounds the number of failed transmissions at n^. The result of this is that 
other than at most failed message transmissions, our protocol enjoys the same throughput 
efficiency of the old Slide protocol. The formal statement of this fact can be found in Theorem 
18.11 in Section [8l and its proof can be found in Section [TOl 

3 The Formal Model 

It will be useful to describe two models in this section, one in the presence of an edge-scheduling 
adversary (all nodes act "honestly"), and one in the presence of an adversary who may "corrupt" 
some of the nodes in the network. In Section [4] we present an efficient protocol ("Slide") that works 
well in the edge-scheduling adversarial model, and we then extend this protocol in Section[8]to work 
in the additional presence of the node-controlling adversary. 
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3.1 The Edge-Scheduling Adversarial Model 

We model a communication network by an undirected graph G = iV^E), where \V\ = n. 
Each vertex (or node) represents a processor that is capable of storing information (in its buffers) 
and passing information to other nodes along the edges. We distinguish two nodes, the sender, 
denoted by 5, and the receiver, denoted by R. In our model, S has an input stream of messages 
{mi,m2, . . . } of uniform size that he wishes to transmit through the network to R. As mentioned in 
the Introduction, the three commodities we care about are Correctness, Throughput, and Processor 
Memory. 

We assume a synchronous network, so that there is a universal clock that each node has access 
tcl^. The global time is divided into discrete chunks, called rounds, during which nodes communicate 
with each other and transfer packets. Each round consists of two equal intervals of unit time called 
stages, so that all nodes are synchronized in terms of when each stage begins and ends. We assume 
that the edges have some fixed capacity P in terms of the amount of information that can be 
transmitted across them per stage. The messages will be sub-divided into packets of uniform size 
P, so that exactly one packet can be transferred along an edge per stagj^. 

The sole purpose of the network is to transmit the messages from S to R, so S is the only 
node that introduces new messages into the network, and R is the only node that removes them 
from the network (although below we introduce a node-controlling adversary who may corrupt the 
intermediate nodes and attempt to disrupt the network by illegally deleting/introducing messages). 
Although the edges in our model are bi-directional, it will be useful to consider each link as consisting 
of two directed edges. Except for the conforming restriction (see below), we allow the edges of our 
network to fail and resurrect arbitrarily. We model this via an Edge- Scheduling Adversary, who 
controls the status of each edge of the network, and can alter the state of any edge at any time. We 
say that an edge is active during a given stage/round if the edge-scheduling adversary allows that 
edge to remain "up" for the entirety of that stage/round. We impose one restriction on the failure 
of edges: 

Definition 3.1. An edge-scheduling adversary is conforming if for every round of the protocol, 
there exists at least one path between S and R consisting of edges that active for the entirety of 
the round. 

For a given round t, we will refer to the path guaranteed by the conforming assumption as the 
active path of round t. Notice that although the conforming assumption guarantees the existence of 
an active path for each round, it is not assumed that any node (including S and R) is aware of what 
that path is. Furthermore, this path may change from one round to the next. The edge-scheduling 
adversary cannot affect the network in any way other than controlling the status of the edges. In 
the next section, we introduce a node-controlling adversary who can take control of the nodes of 
the networHll^. 



^Although synchronous networks are difficult to realize in practice, we can further relax the model to one in which 
there is a known upper-bound on the amount of time an active edge can take to transfer a packet. 

^°Our protocol for the node-controlling adversarial model will require the packets to include signatures from a 
cryptographic signature scheme. The security of such schemes depend on the security parameter k, and the size of 
the resulting signatures have size 0(k). Additionally, error-correction will require packets to carry with them an 
index of O(logn) bits. Therefore, we assume that P > (fc-|~logn), so that in each time step a complete packet (with 
signature and index) can be transferred. 

^^The distinction between the two kinds of adversaries is made solely to emphasize the contribution of this paper. 
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3.2 The Node-Controlling + Edge- Scheduling Adversarial Model 

This model begins with the edge-scheduling adversarial model described above, and adds a poly- 
nomially bounded Node-Controlling Adversary that is capable of corrupting nodes in the network. 
The node-controlling adversary is malicious, meaning that the adversary can take complete control 
over the nodes he corrupts, and can therefore force them to deviate from any protocol in whatever 
manner he likes. We further assume that the adversary is dynamic, which means that he can corrupt 
nodes at any stage of the protocol, deciding which nodes to corrupt based on what he has observed 
thus faiEl. For a thorough discussion of these notions, see [Tn| and references therein. 

As in Multi-Party Computation (MPC) literature, we will need to specify an "access-structure" 
for the adversary. 

Definition 3.2. A node-controlling adversary is conforming if he does not corrupt any nodes who 
have been or will be a part of any round's active path. 

Apart from this restriction, the node-controlling adversary may corrupt whoever he likes (i.e. it 
is not a threshold adversary) . Note that the conforming assumption implicitly demands that S and 
R are incorruptible, since they are always a part of any active path. Also, this restriction on the 
adversary is really more a statement about when our results remain valid. This is similar to e.g. 
threshold adversary models, where the results are only valid if the number of corrupted nodes does 
not exceed some threshold value t. Once corrupted, a node is forever considered to be a corrupt 
node that the adversary has total control over (although the adversary may choose to have the node 
act honestly). 

Notice that because correctness, throughput, and memory are the only commodities that our 
model values, an honest-but-curious adversary is completely benign, as privacy does not need to be 
protecteclll (indeed, any intermediate node is presumed to be able to read any packet that is passed 
through it). Our techniques for preventing/detecting malicious behavior will be to incorporate a 
digital signature scheme that will serve the dual purpose of validating information that is passed 
between nodes, as well as holding nodes accountable for information that their signature committed 
them to. 

We assume that there is a Public-Key Infrastructure (PKI) that allows digital signatures. In 
particular, before the protocol begins we choose a security parameter k sufficiently large and run a 
key generation algorithm for a digital signature scheme, producing n = \G\ (secret key, verification 
key) pairs {skiy,vk]\f). As output to the key generation, each processor € G is given its own 
private signing key skjy and a list of all n signature verification keys vk^;^ for all nodes N £ G. In 
particular, this allows the sender and receiver to sign messages to each other that cannot be forged 
(except with negligible probability in the security parameter) by any other node in the system. 



Edge-scheduling adversaries (as described above) are commonly used to model edge failures in networks, while the 
contribution of our paper is in controlling a node-controlling adversary, which has the ability to corrupt the nodes of 
the network. 

^^Although the node-controlling adversary is dynamic, he is still constrained by the conforming assumption. 
Namely, the adversary may not corrupt nodes that have been, or will be, part of any active path connecting sender 
and receiver. 

^''if desired, privacy can be added trivially by encrypting all packets. 
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4 Routing Protocol in the Edge- Scheduling Adversarial Model 



In this section we formally describe our edge-scheduling protocol, which is essentially the "Slide" 
protocol of [2|. 

4.1 Definitions and High-Level Ideas 

The goal of the protocol is to transmit a sequence of messages {mi, m2, . . . } of uniform size from 
the sender S to the receiver R (refer to Section ISTTl for a complete description of the model). Each 
node will maintain a stack (i.e. FILO buffers) along each of its (directed) edges that can hold up to 
2n packets concurrently. To allow for packets to become stuck in the buffers, we will utilize error- 
correction (see e.g. [lO]). Specifically, the messages {mi,m2, ■ ■ ■} are converted into codewords 
{ci,C2,...}, allowing the receiver to decode a message provided he has received an appropriate 
number (depending on the information rate and error-rate of the code) of bits of the corresponding 
codeword. In this paper, we assume the existence of a error-correcting code with information rate 
a and error rate A. 

As part of the setup of our protocol, we assume that the messages {mi,m2, ■ ■ ■} have been 
partitioned to have uniform size M = (recall that P is the capacity of each edge and a and 

A are the parameters for the error-correction code). The messages are expanded into codewords, 
which will have size C = ^ = . The codewords are then divided into p = ^j- packets of size 
P. We emphasize this quantity for later use: 



Note that the only "noise" in our network results from undelivered packets or out-dated packets (in 
the edge-scheduling adversarial model, any packet that R receives has not been altered). Therefore, 
since each codeword consists of D = ^ packets, by definition of A, if R receives (1 — \)D = 

(1 — A) (^^^ packets corresponding to the same codeword, he will be able to decode. We emphasize 
this fact: 



Fact 1. If the receiver has obtained D — = (1 — A) j packets from any codeword, he 
will be able to decode the codeword to obtain the corresponding message. 

Because our model allows for edges to go up/down, we force each node to keep incoming and 
outgoing buffers for every possible edge, even if that edge isn't part of the graph at the outset. We 
introduce now the notion of height of a buffer, which will be used to determine when packets are 
transferred and how packets are re-shuffled between the internal buffers of a given node between 
rounds. 

Definition 4.1. The height of an incoming/outgoing buffer is the number of packets currently 
stored in that buffer. 

The presence of an edge-scheduling adversary that can force edges to fail at any time complicates 
the interaction between the nodes. Note that our model does not assume that the nodes are aware 
of the status of any of its adjacent edges, so failed edges can only be detected when information 
that was supposed to be passed along the edge does not arrive. We handle potential edge failures 
as follows. First, the incoming/outgoing buffers at either end of an edge will be given a "status" 



D := — — = number of packets per codeword. 



(1) 
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(normal or problem). Also, to account for a packet that may be lost due to edge failure during 
transmission across that edge, a node at the receiving end of a failed edge may have to leave room 
in its corresponding incoming buffer. We refer to this gap as a ghost packet, but emphasize that 
the height of an incoming buffer is not affected by ghost packets (by definition, height only counts 
packets that are present in the buffer). Similarly, when a sending node "sends" a packet across an 
edge, it actually only sends a copy of the packet, leaving the original packet in its outgoing buffer. 
We will refer to the original copy left in the outgoing buffer as a flagged packet, and note that flagged 
packets continue to contribute to the height of an outgoing buffer until they are deleted. 

The codewords will be transferred sequentially, so that at any time, the sender is only inserting 
packets corresponding to a single codeword. We will refer to the rounds for which the sender is 
inserting codeword packets corresponding to the i*^ codeword as the i^^ transmission. Lemma 
16.151 below states that after the sender has inserted D — 2n^ packets corresponding to the same 
codeword, the receiver can necessarily decode. Therefore, when the sender has inserted this many 
packets corresponding to codeword bi, he will clear his outgoing buffers and begin distributing 
packets corresponding to the next codeword 

4.2 Detailed Description of the Edge-Scheduling Protocol 

We describe now the two main parts of the edge-scheduling adversarial routing protocol: the 
Setup and the Routing Phase. For a formal presentation of the pseudo-code, see Section [5l 

Setup. Each internal (i.e. not S or R) node has the following buffers: 

1. Incoming Buffers. Recall that we view each bi-directional edge as consisting of two directed 
edges. Then for each incoming edge, a node will have a buffer that has the capacity to hold 2n 
packets at any given time. Additionally, each incoming buffer will be able to store a "Status" 
bit, the label of the "Last-Received" packet, and the "Round- Received" index (the round in 
which this incoming buffer last accepted a packet, see Definition 16.51 below) . The way that this 
additional information is used will be described in the "Routing Rules for Receiving Node" 
section below. 

2. Outgoing Buffers. For each outgoing edge, a node will have a buffer that has the capacity to 
hold 2n packets at any given time. Like incoming buffers, each outgoing buffer will also be 
able to store a status bit, the index label of one packet (called the "Flagged" packet), and a 
"Problem-Round" index (index of the most recent round in which the status bit switched to 

1). 

The receiver will only have incoming buffers (with capacity of one) and a large Storage Buffer that 
can hold up to D packets. Similarly, the sender has only outgoing buffers (with capacity 2n) and 
the input stream of messages {mi, m2, • • • } which are encoded into the codewords and divided into 
packets, the latter then distributed to the sender's outgoing buffers. 

Also as part of the Setup, all nodes learn the relevant parameters (P, n. A, and a). 

Routing Phase. As indicated in Section ISTTl we assume a synchronous network, so that there are 
well-defined rounds in which information is passed between nodes. Each round consists of two units 
of time, called Stages. The formal treatment of the Routing Phase can be found in the pseudo-code 
of Section m Informally, Figure [2] below considers a directed edge E{A, B) from A (including A = S) 
to B (including B = R), and describes what communication each node sends in each stage. 
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Stage 
1 


A 


Hb ■■= 
Round 


B 


Ha := Height of buffer along E{A, B) 
Height of flagged p. (if there is one) — 
Round prev. packet was sent 


Height of buffer along E{A,B) 
prev. packet was received 




Send packet if: 






2 


• Ha>Hb OR - 








• B didn't rec. prev. packet sent 







Figure 2: Description of communication exchange along directed edge E{A,B) during the Routing 
Phase of any round. 

In addition to this communication, each node must update its internal state based on the 
communication it receives. In particular, from the communication A receives from B in Stage 1 of 

any round, A can determine if B has received the most recent packet A sent. If so, A will delete 
this packet and switch the status of the outgoing buffer along this edge to "normal." If not, A will 
keep the packet as a flagged packet, and switch the status of the outgoing buffer along this edge 
to "problem." At the other end, if B does not receive ^'s Stage 1 communication or B does not 
receive a packet it was expecting from A in Stage 2, then B will leave a gap in its incoming buffer 
(termed a "ghost packet") and will switch this buffer's status to "problem." On the other hand, if 
B successfully receives a packet in Stage 2, it will switch the buffer back to "normal" status. 

Re-Shuffle Rules. At the end of each round, nodes will shuffle the packets they are holding 
according to the following rules: 

1. Take a packet from the fullest buffer and shuffle it to the emptiest buffer, provided the differ- 
ence in height is at least two (respectively one) when the packet is moved between two buffers 
of the same type (respectively when the packet moves from an incoming buffer to an outgoing 
buffer). Packets will never be re-shuffled from an outgoing buffer to an incoming buffer. If 
two (or more) buffers are tied for having the most packets, then a packet will preferentially 
be chosen from incoming buffers over outgoing buffers (ties are broken in a round-robin fash- 
ion). Conversely, if two (or more) buffers are tied for the emptiest buffer, then a packet will 
preferentially be given to outgoing buffers over incoming buffers (again, ties are broken in a 
round-robin fashion) . 

2. Repeat the above step until the difference between the fullest buffer and the emptiest buffer 
does not meet the criterion outlined in Step 1. 

Recall that when a packet is shuffled locally between two buffers, packets travel in a FILO manner, 
so that the top-most packet of one buffer is shuffled to the top spot of the next buffer. When an 
outgoing buffer has a flagged packet or an incoming buffer has a ghost packet, we use instead the 
following modifications to the above re-shuffle rules. Recall that in terms of measuring a buffer's 
height, flagged packets are counted but ghost packets are not. 

- Outgoing buffers do not shuffle flagged packets. In particular, if Rule 1 above selects to transfer 
a packet from an outgoing buffer, the top-most non-flagged packet will be shuffled. This may 
mean that a gap is created between the flagged packet and the next non-flagged packet. 
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- Incoming buffers do not re-shuffle ghost packets. In particular, ghost packets will remain in 
the incoming buffer that created them, although we do allow ghost packets to slide down 
within its incoming buffer during re-shuffling. Also, packets shuffled into an incoming buffer 
are not allowed to occupy the same slot as a ghost packet (they will take the first non-occupied 
slotB 

The sender and receiver have special rules for re-shuffling packets. Namely, during the re-shuffle 
phase the sender will ffll each of his outgoing buffers (in an arbitrary order) with packets correspond- 
ing to the current codeword. Meanwhile, the receiver will empty all of its incoming buffers into its 
storage buffer. If at any time R has received enough packets to decode a codeword bi (Fact 1 says 
this amount is at most D — 6n^), then R outputs message rrii and deletes all packets corresponding 
to codeword bi from its storage buffer that he receives in later rounds. 

4.3 Analysis of the Edge-Scheduling Adversarial Protocol 

We now evaluate our edge-scheduling protocol in terms of our three measurements of perfor- 
mance: correctness, throughput, and processor memory. The throughput standard expressed in 
Theorem 14.21 below will serve an additional purpose when we move to the node-controlling ad- 
versary setting: The sender will know that malicious activity has occurred when the throughput 
standard of Theorem 14.21 is not observed. Both of the theorems below will be proved rigorously in 
Sections [6] and [7l after presenting the pseudo-code in Section [5l 

Theorem 4.2. Each message mi takes at most 3D rounds to pass from the sender to the receiver. 
In particular, after 0{xD) rounds, R will have received at least 0{x) messages. Since each message 
has size M =^Pn^ =0{n^) and D=^=0{n^), after 0{x) rounds, R has received 0{x) hits of 
information, and thus our edge- scheduling adversarial protocol enjoys a linear throughput rate. 

The above theorem implicitly states that our edge-scheduling protocol is correct. For complete- 
ness, we also state the memory requirements of our edge-scheduling protocol, which is bottle-necked 
by the O(n^) packets that each internal node has the capacity to store in its buffers. 

Theorem 4.3. The edge-scheduling protocol described in Section (and formally in the pseudo- 
code of Sectionl^ requires at most 0(n^ log n) bits of memory of the internal processors. 



Note that because ghost packets do not count towards height, there appears to be a danger that the re-shufBe rules 
may dictate a packet gets transferred into an incoming buffer, and this packet either has no place to go (because the 
ghost packet occupies the top slot) or the packet increases in height (which would violate Claim [6]4] below). However, 
because only incoming buffers are allowed to re-shufHe packets into other incoming buffers, and the difference in 
height must be at least two when this happens, neither of these troublesome events can occur. 



17 



5 Pseudo-Code for the Edge-Scheduling Adversarial Protocol 



Setup 








DEFINITION OF VARIABLES: 




01 


n := 


= Number of nodes in G; 




02 


D : 


_ 6n^ . 

^ A ' 




03 


T := 


: Transmission index; 




04 


t := 


= Stage/Round index; 




05 


P :-- 


= Capacity of edge (in bits); 




06 


for 


every N £ G 




07 




for every outgoing edge E{N, B) € G,B ^ S and N ^ R 

OUT € [2n] X {0, 1}^; ## Outgoing Buffer able to hold 2n packets 


08 




09 




P€{0, iru±; 


## Copy of packet to be sent 


10 




sfee{0,l}; 


## Status bit 


11 




d € {0, 1}; 


## Bit indicating if a packet was sent in the previous round 


12 




FR e [0..6D] U±; 


## Flagged Round (index of round A'' first tried to send p to B) 


13 




H € [0..2n]; 


## Height of OUT. Also denoted Hqut when there's ambiguity 


14 




Hfp G [1..2n] U_L; 


## Height of Flagged Packet 


15 




RR e [-1..6D] U_L; 


4^4i- Round Received index (from adjacent incoming buffer) 


16 




HiN € [0..2n] U _L; 


## Height of incoming buffer of B 


17 




for every incoming edge E{A, N) & G, A ^ R and N ^ S 


18 




IN € [2n] X {0,ir; 


## Incoming Buffer able to hold 2n packets 


19 




PG{0,1}^U±; 


## Packet just received 


20 




she {0,1}; 


4tit Status bit 


21 




RR e [-1..6D]; 


## Round Received (index of round N last rec'd a p. from A) 


22 




H € [0..2n]; 


## Height of IN. Also denoted i?/jv when there's ambiguity 


23 




Hgp e [1..2n] U ±; 


## Height of r.host Packet 


24 




Hour e [0..2n] U ±; 


## Height of outgoing buffer, or height of Flagged Packet of A 


25 




sbouT e {0, 1}; 


## Status Bit of outgoing buffer of A 


26 




FR € [0..6D] U _L; 


## Flagged Round index (from adjacent outgoing buffer) 




INITIALIZATION OF VARIABLES: 




27 


for 


every N E G 




28 




for every incoming edge E{A, N) £ G,A ^ R and N S 


29 




Initialize IN; 


## Set each entry in IN to ± 


30 




P,FR,Hgp = ±; 




31 




sb, sbouT, H, Hour = 0; ii-R = 


-1; 


32 




for every outgoing edge E{N, B) £ G,B ^ S and N ^ R, 


33 




Initialize OUT; 


## Set each entry in OUT to ± 


34 




P,Hfp,RR,FR = _L; 




35 




sb, d, H,HiN = 0; 




En 


d Setup 







Figure 3: Pseudo-Code for Internal Nodes' Setup for the Edge-Scheduling Adversarial Model 
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Sender and Receiver's Additional Setup 



DEFINITION OF ADDITIONAL VARIABLES FOR SENDER: 

36 Ai ~ {mi, 7712, . . . } = Input Stream of Messages; 

37 K e [0..D] = Number of packets corresponding to current codeword the sender has knowingly inserted; 

INITIALIZATION OF SENDER'S VARIABLES: 

38 Distribute Packets; ## See Figure [6] 

39 K = 0; 

DEFINITION OF ADDITIONAL VARIABLES FOR RECEIVER: 

40 Ir G [D] X ({0, 1}^ U _L) = Storage Buffer to hold packets corresponding to current codeword; 

41 K G [0..-D] := Number of packets received corresponding to current codeword; 

INITIALIZATION OF RECEIVER'S VARIABLES: 

42 K = 0; 

43 Initialize Ir; ## Sets each element of Ir to -L 



End Sender and Receiver's Additional Setup 

Figure 4: Additional Code for Sender and Receiver Setup 



Transmission T 




01 for every N E G 




02 


for every t < 2 * (3D) ## The factor of 2 is for the 2 stag 


es per round 


03 


if t (mod 2) = then: ## STAGE 1 




04 


for every outgoing edge E{N, B) e G,N ^ R,B ^ S 




05 


if Hfp ^ ±: send (H,±,±); else: send {H - 1, Hfp, FR); 




06 


receive {Hin , RR); 




07 


Reset Outgoing Variables; 




08 


for every incoming edge E{A, N)(^G,N^S,A^R 




09 


send {H,RR); 




10 


sbouT = G;FR = ±; 




11 


receive {H, _L, _L) or {H, Hfp, FR); ## If = _L or Fi? > RR, set sbo 


i7T=l; and 




## Hout=Hfp; O.W. set Hout= 


-H; sbouT=0; 


12 


else if t (mod 2) = 1 then: ## STAGE 2 




13 


for every outgoing edge E{N, B) e G,N ^ R,B ^ S 




14 


if Him -L then: Received B's info. 




15 


Create Flagged Packet; 




16 


if (s6=l or (s6=0 and Hour > Hin)) then: 




17 


Send Packet; 




18 


for every incoming edge E{A, N) £ G, N ^ S, A R 




19 


Receive Packet; 




20 


if N i {5, R} then: Re-Shuffle; 




21 


else if N — R then: Receiver Re-Shuffle; 




22 


else if A'^ = 5* then: Sender Re-Shuffle; 




23 


if t = 2(373) — 1 then: End of Transmission Adjustments; 




End 


Transmission T 





Figure 5: Routing Rules I for Edge-Scheduling Adversarial Model 
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a = U; 
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## Didn t receive cont. 01 packet receipt 
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s6 = 1; 




29 


if RR # ±: 




Qfl 

oU 


11 _L 7= -T -Tt \ Itri. 


7^7^ B rec'd most recently sent packet 


oi 


11 iV =0 trien: k = k + i; 




32 


OUT[i/Fp] = -L; Fill Gap; 


## Remove p from OUT, shifting 






## down packets on top of p if necessary 


33 


r H,p, tlpp = J-\ SO = \J\ tl =11 — i; 




34 


11 ± 7= ilit < r H and ± 7^ -Hfp < ^i: 


## B did not receive most recently sent packet 


35 


Elevate Flagged Packet; 


## Swap Oy^T[H] and Oy^T[HFp]\ Set Hpp = H; 


36 


1 7711 J n I J. 

Create Flagged Packet 




/ 


11 oo — u ana n ^ n I M • 


7^7^ Normal Status, will send top packet 


QQ 
00 


p = <J\j \ [rl\\ riFp = ti \ r H = t; 




Oi) 


Send Packet 




/in 


d= 1; 




41 


send {p,FR); 






Receive Packet 






receive (p, FR); 




44 


if Hour = -L: 


## Didn't Rec. A's height info. 


40 


sb= 1; 




40 


if Hop > H or {Hop = ± and H < 2n): Hap = H + 1; 


4/ 


else if sbouT = 1 or Hour > H: 


## A packet should've been sent 


4o 


ifp = ±: 


i^i^ Packet wasn't rec'd 


4y 


s6 = 1; 




OU 


if i/cp > or (Hgp = ± and < 2n): i?GP 


= H+1; 


Oi 


else if RR < FR: 


## Packet was rec'd and should keep it 


OZ 


if Hgp = Hgp = H + 1; 


## If no slot is saved for p, put it on top 


Oo 


sb = 0; \N[Hgp]=p; H = H + 1; Hop = -L; RR 


= t; 


04 


else: 


## Packet was rec'd, but already had it 


00 


sb = 0; Fill Gap; Hgp = -L; 


i^i^ See comment about Fill Gap on line 57 below 


od 


else: 


i^i^ A packet should XOT have been sent 


57 


sb = 0; Fill Gap; Hgp = -L; 


## If packets occupied slots above the 






## Ghost Packet, then Fj« Gap will Slide 






## those packets down one slot 


58 


End of Transmission Adjustments 




59 


for every outgoing edge £;(iV, B) £ G, N ^ R, B ^ S: 




60 


if Hfp 7^ ±: 




61 


OUT[Hfp] = -L; R// Gap; 


## Remove any flagged packet p from OUT, shifting 






## down packets on top of p if necessary 


62 


d, 6-6 = 0; FR,Hfp,P = ±; H = H - 1; 




63 


for every incoming edge A'") G G, N ^ S, A R: 




64 


Hgp = ±; sb = 0; i?Ji = -1; Fill Gap; 




65 


if N = S then: Distribute Packets; 





Figure 6: Routing Rules for Edge-Scheduling Adversarial Model (continued) 



20 



71 Re-Shuffle 
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(M, Bp) = Find Maximum Buffer 


itit i^ode iv nncis its luuest uuiier t>F witn neignt iw , 






## breaking ties by 1) selecting incoming buffers over 






## outgoing buffers, then 2) Round-Robin 


73 


(m, Bt) = Find Minimum Buffer 


## Node A'' finds its emptiest buffer Bt with height m, 






## breaking ties by 1) selecting outgoing buffers over 






## incoming buffers, then 2) Round-Robin 


74 


if Packet Should Be Re-Shuffled: 


## A packet should be re-shuffled if M — m > 1 or 






„ „ , . , , f Bp is an Inc. Buffer 1 






## M - m = 1 and <^ ^ \ 


75 


Adjust Heights 


Adjust -A^, to ciccoiint for Ghost, Plugged packets. 


76 


SIGn,n = SIGn,n + (M - m - 1); 


-ff--ff- Wiliy UbcU iUi 1 11UU.C UUllll . cU^c oLIIcU. I ]Ji U lULUl 


77 


Shuffle Packet 




78 


Re-Shuffle 




79 


Adjust Heights 




80 


if Bp is an Out. Buffer and Hfp > Hqut'- 


//- tipp and tiQUT reier to -dj? s into, li true, 


81 


M = M-1; 


then a Flagged packet is top-most non-null packet 


82 


if Bf is an Inc. Buffer and \N[Hin + 1] -L: 


## IN and Bin refer to Bf's info. If true, 


83 


M = M+ 1; 


-^-^ then there is a Ghost Packet creating a gap 


84 


if Bt is an Out. Buffer and OUT[Hout] = -L: 


ff^ff^ ^-'u 1 dna ^iQUT reiei to dt s inio. ii tiue. 


85 


m = m — 1; 


then there is a Flagged packet creating a gap 


86 


if Bt is an Inc. Buffer and Hop -L: 


ff-ff- ^GP cLLiu I N leiei lu dt >^ luio. ii Line, 


87 


m = m + 1; 


then there is a Ghost Packet creating a gap 


88 


Shuffle Packet 




89 


BT[m + 1] = Bf[M]; 




90 


Bf[M] = ±; 




91 


Hbt ~ + 1; 


-jfii Hbt is the height of Bt 


92 


Hbp = Hbp — 1; 


## Hbp is the height of Bf 


93 


if Bf is an Inc. Buffer and _L ^ Hqp > Hin, then: ## Hqp and Him refer to Bf's info. Since Bf lost a 


94 


Hgp = Hin + 1; 


## packet, slide Ghost Packet down into top slot 


95 


Sender Re-Shuffle 




96 


Fill Packets; 


## Fills each outgoing buffer with codeword packets not 






## yet distributed, adjusting each Hqut appropriately 


97 


Receiver Re-Shuffle 




98 


for every incoming edge E{A,R) £ G: 


## Reset R's Inc. Buffer to be open 


99 


if H/jv > 0: 


## ii rec'd a packet along this edge this round 


100 


if /A'^[l] is a packet for current codeword: 


## Also, see comments on 104 below 


101 


Ir[k]^ IN[1]; K = K + 1; 




102 


Hin = 0; /iV[l] = -L; Hqp = -L; 




103 


if K > D - 3n^ then: 


## R can decode by Fact 1 


104 


Decode and output message; 


## Also, only keep codeword packets corresponding 






## to next message in future rounds 



Figure 7: Re-Shuffle Rules for both Edge-Scheduling and (Node-Controlling + Edge-Scheduling) 
Protocols 
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6 Edge- Scheduling Adversary Model: 
Proofs of Lemmas and Theorems 



Before proving the main two theorems for the edge-scheduling adversarial protocol (Theorems 
14.21 and l4.3p . we will first state and prove a sequence of claims that follow immediately from the 
Routing and Re-Shuffle Rules of Section UTTl We have included pseudo-code in Section [H and when 
appropriate the proofs will refer to specific line numbers in the pseudo-code. In particular, we will 
reference a line in pseudo-code by writing (X.YY), where X refers to the Figure and YY to the line 
number. We pushed the claims and proofs that rely heavily on the pseudo-code (but are unlikely 
to add insight) to Section [7] so as not to distract the reader with the gory details of these proofs. 
Logically, these claims need to be proven first as the claims and proofs below will rely on them 
(even though Section [7] appears below, the proofs there do not rely on proofs here, so there is no 
danger of circularity). 

We state and prove here the claims that will lead to Theorem 14.21 and Theorem I4.3[ 

Claim 6.1. The capacity of the internal buffers of the network (not counting S or R's buffers) is 
4n(n - 2)2. 

Proof. Each node has {n — 2) outgoing buffers (one to each node except itself and S, [3l07) and 
(n — 2) incoming buffers (one from each node except itself and i?,[3ll7), and thus a total of 2(n — 2) 
buffers. Each of these buffers has capacity 2n (Lemma 17. 11 parts 5, 6, and 9), and there are n — 2 
internal nodes, so the internal buffer capacity of the network is 4n(n — 2)^. ■ 

Claim 6.2. The maximum amount of potential in the internal buffers of the network at any time 
ts 2n(2n + l)(n - 2f . 

Proof. A buffer contributes the most to network potential when it is full, in which case it contributes 
Yld=i ^ ~ "'(^'^ + !)■ Since there are 2(n — 2) buffers per internal node, and n — 2 internal nodes, 
the maximum amount of potential in the internal buffers is as claimed. ■ 

We define the height of a packet in an incoming/outgoing buffer to be the spot it occupies in that 
buffer. 

Claim 6.3. After re-shuffling, (and hence at the very end/beginning of each round), all of the buffers 
of each node are balanced. In particular, there are no incoming buffers that have height strictly 
bigger than any outgoing buffers, and the difference in height between any two buffers is at most 
one. 

Proof of Claim We prove this using induction (on the round index), noting that all buffers 
are balanced at the outset of the protocol (lines ([31.29) and ([3133)). Consider any node N in the 
network, and assume that its buffers are all balanced at the end of some round t. We need to show 
the buffers of N will remain balanced at the end of the next round t + 1. Let Bi and B2 denote 
any two buffers of A^, and let hi be the variable denoting the height of Bi and /12 the height of B2. 
Suppose for the sake of contradiction that /ii > /i2 + 2 at the end of round t + 1 (after re-shuffling). 
Let H denote the height of the maximum buffer in at the end oit + l,soH>hi > h2 + 2. Also 

bee Definition [QOl 
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let h denote the height of the minimum buffer in N at the end oft + 1, so/i</i2<^^ — 2. But 
then Re-Shuffle Rules dictate that A'^ should've kept re-shuffling ([7172-74), a contradiction. 

Similarly, assume for contradiction that there exists an incoming buffer whose height /12 is bigger 
than that of some outgoing buffer that has height hi. Let H and h be as defined above, so we have 
that h < hi < h2 < H. In the case that /12 = Re-ShufHe Rules ([71.72) guarantee that an 
incoming buffer will be selected to take a packet from. Also, if h = hi, then Re-Shuffle Rules ([3.73) 
guarantee that an outgoing buffer will be chosen to give a packet to. Therefore, in this case a packet 
should have been re-shuffled ([374), and hence we have contradicted the fact that we are at the end 
of the Re-ShufHe phase of round t. On the other hand, if /i 7^ /ii or i7 / /12, then H — h > 2, and 
again Re-Shuffling should not have terminated ([374). ■ 

The following observation is the formalization of the concept of packets "flowing downhill" that was 
introduced in Section 14.11 

Claim 6.4. Every packet is inserted into one of the sender's outgoing buffers at some initial height. 
When (a copy of) the packet goes between any two buffers Bi / B2 (either across an edge or locally 
during reshuffling), its height in B2 is less than or equal to the height it had in Bi. If Bi = B2, 
the statement remains true EXCEPT for on line (ElSSj. 

Proof. See Section [7l where we restate and prove this in Lemma FT. Ill ■ 

Definition 6.5. We will say that a packet is accepted by a buffer B in round t if i3 receives and 
stores that packet in round t, either due to a packet transfer or re-shuffling (as on ([6l53) or ([389)). 

Definition 6.6. We say that the sender inserts a packet into the network in round t if any internal 
node (or R) accepts the packet (as in Definition 16. 5p in round t. Note that this definition does not 
require that S receives the verification of receipt (i.e. that S receives the communication on ([5l06) 
indicating RR > FR), so S may not be aware that a packet was inserted. 

Notice that in terms of transferring packets, the above definition distinguishes between the case that 
a packet is accepted by B in round t (as defined above) and the case that a packet arrives at B in 
round t but is deleted by B (by failing the conditional statement on line ([6l51)). As emphasized in 
the Introduction, correctness and throughput rate are two of the three commodities with which we 
will evaluate a given routing protocol. In our protocol, we will need to show that packets are not 
lost en route from S" to i? to ensure correctness, and meanwhile we will want to show that packets 
are not (overly) duplicated (since transferred packets are actually copies of the original, some packet 
duplication is necessary) to allow a "fast" throughput rate. The following two claims guarantee that 
packet duplication won't become problematic while simultaneously guaranteeing that packets are 
never deleted completely (except by R). 

Claim 6.7. Before the end of transmission T, any packet that was inserted into the network during 
transmission T is either in some buffer (perhaps as a flagged packet) or has been received by R. 

Proof. See Section [7l where we restate and prove this in Lemma FT. 121 ■ 

Claim 6.8. Not counting flagged packets, there is at most one copy of any packet in the network at 
any time (not including packets in the sender or receiver's buffers). Looking at all copies (flagged 
and un- flagged) of any given packet present in the network at any time, at most one copy will ever 
be accepted (as in Definition \6. 5)) by another node. 
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Proof. See Section [7l where we restate and prove this in Lemma [7. 131 



The following claim won't be needed until we introduce the protocol for the (Node-Controlling 
+ Edge-Scheduling) adversarial model, but follows from the Routing Rules outlined in Section [4m 

Claim 6.9. At any time, an outgoing buffer has at most one flagged packet. 

Proof. See Section [7l where we restate and prove this in Corollary 17. 81 ■ 

The following definition formalizes the notion of "potential," and will be necessary to prove 
throughput performance bounds. 

Definition 6.10. For any buffeiEl B ^ S,R that has height h at time t, define the potential of B 
at time t, denoted by $f , to be: 

i=l 

For any internal node € "P \ {R, S}, define the node's potential to be the sum of its buffer's 
potentials: 

Buffers B oi N 

Define the network potential at time t to be the sum of all the internal buffers' potentials: 

:= E < 

NeV\{R,S} 

It will be useful to break an internal node's potential into two parts. The first part, which we will 
term packet duplication potential, will be the sum of the heights of the fiagged packets in the node's 
outgoing buffers that have already been accepted by the neighboring node (as in Definition 16.5(1 . 
Recall that a flagged packet is a packet that was sent along an outgoing edge, but the sending node 
is maintaining a copy of the packet until it gets confirmation of receipt. Therefore, the contribution 
of packet duplication potential to overall network potential is the extraneous potential; it represents 
the over-counting of duplicated packets. We emphasize that not all fiagged packets count towards 
packet-duplication potential, since packets are fiagged as soon as the sending node determines a 
packet should be sent (see line [61.38), but the fiagged packet's height does not count towards packet 
duplication potential until the receiving node has accepted the packet as on line ([61.53) (which may 
happen in a later round or not at all). The other part of network potential will be termed non- 
duplicated potential, and is the sum of the heights of all non-fiagged packets together with fiagged 
packets that have not yet been accepted. Note that the separation of potential into these two parts 
is purely for analysis of our protocol, indeed the nodes are not able to determine if a given fiagged 
packet contributes to packet duplication or non- duplicated potential. For convenience, we will often 
refer to (network) non-duplicated potential simply as (network) potential (the meaning should be 
clear from context). 

Notice that when a node accepts a packet, its own (non-duplicated) potential instantaneously 
increases by the height that this packet assumes in the corresponding incoming buffer. Meanwhile, 
the sending node's non- duplicated potential drops by the height that the packet occupied in its 

^^Packets in one of the sender or receiver's buffers do not count towards potential. 
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outgoing buffer, and there is a simultaneous and equivalent increase in this sending node's packet 
duplication potential. Separating overall network potential into these two categories will be necessary 
to state and prove the following Lemma: 

Lemma 6.11. Every change in network potential comes from one of the following 3 events: 

1. S inserts a packet into the network. 

2. R receives a packet. 

3. A packet that was sent from one internal node to another is accepted; the verification of packet 
receipt is received by the sending node; a packet is shuffled between buffers of the same node; 
or a packet is moved within a buffer. 

Furthermore, changes in network potential due to item 1 ) are strictly non-negative and changes due 
to item 2) are strictly non-positive. Also, changes in network non- duplicated potential due to 
item 3) are strictly non-positive. Finally, at all times, network packet duplication potential is 
bounded between zero and 2n'^ — S-n? + 8n. 

Proof. Since network potential counts the heights of the internal nodes' buffers, it only changes 
when these heights change, which in turn happens exclusively when there is packet movement. By 
reviewing the pseudo-code, we see that this happens only on lines ([6132), JBlSS), ([6l53), JBlSS), 
dSlSr), JSei), (I6l64), and ([389-90). Each of these falls under one of the three items listed in 
the Lemma, thus proving the first statement in the Lemma. That network potential changes due 
to packet insertion by S are strictly non-negative is obvious (either the receiving node's potential 
increases by the height the packet assumed, as on JBlSS), or the receiving node is R and the 
packet does not contribute to potential). Similarly, that potential change upon packet receipt by 
R is strictly non-positive is clear, since packets at R do not count towards potential (see Definition 
IG.lOp . Also, since only flagged packets (but not necessarily all of them) contribute to network packet 
duplication potential, the biggest it can be is the maximal number of flagged packets that can exist 
in the network at any given time, times the maximum height each flagged packet can have. By 
Claim WM there are at most (n — 2)^ flagged packets in the network at any given time, and each one 
has maximal height 2n (Lemma l7.ll part 9), so network packet duplication potential is bounded by 
2n^ - Sn^ + 8n. 

It remains to prove that changes in network non-duplicated potential due to item 3) are strictly 
non-positive. To do this, we look at all lines on which there is packet movement, and argue each 
will result in a non-positive change to non-duplicated potential. Clearly potential changes on lines 
16132), 16155), 16157), ([6161), and ([6164) are non-positive. Also, if ([1135) is reached, if R has 
already accepted the packet, then that packet's potential will count towards duplicated potential 
within the outgoing buffer, and so the change in potential as on ([6135) will not affect non-duplicated 
potential. If on the other hand R has not already accepted the packet, then the flagged packet still 
counts towards non-duplication potential in the outgoing buffer. Since the result of ([6l35) is simply 
to swap the flagged packet with the top packet in the buffer, the net change in non-duplication 
potential is zero. That changes in potential due to re-shuffling packets ([7189-90) are strictly non- 
positive follows from Claim 16.41 It remains to check the cases that a packet that was transferred 
between two internal nodes is accepted ([6l53). Notice that upon receipt there are two changes to 
network non-duplicated potential: it increases by the height the packet assumes in the incoming 
buffer it arrived at ([6l53), and it decreases by the height the packet had in the corresponding 
outgoing buffer (this decrease is because the flagged packet in the outgoing buffer will count towards 
packet duplication potential instead of non-duplicated potential the instant the packet is accepted). 
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The decrease outweighs the increase since the packet's height in the incoming buffer is less than or 
equal to the height it had in the corresponding outgoing buffer (Claim [6^ . ■ 

The following Lemma will be useful in bounding the number of rounds in which no packets are 
inserted. We begin with the following definition: 

Definition 6.12. The sender is blocked from inserting any packets in some round t if the sender is 
not able to insert any packets in t (see Definition 16. 6p . Let /3t denote the number of rounds in a 
transmission T that the sender was blocked. 

Lemma 6.13. // at any point in any transmission T, the number of blocked rounds is 13-y, then there 
has been a decrease in the network's non- duplicated potential by at leas^\ n(3i. 

The intuition of the proof is to argue that each blocked round creates a drop in non-duplicated 
potential of at least n as follows. If the sender is blocked from inserting a packet, the node N 
adjacent to the sender (along the active honest path) will necessarily have a full incoming buffer 
along its edge to the sender. By the fact that buffers are balanced (Lemma 16. 3|) . this implies that 
all of A^'s outgoing buffers are also full. Meanwhile, at the opposite end of the active honest path, 
the node adjacent to the receiver will necessarily send a packet to the receiver if there is anything 
in its outgoing buffer along this edge, and this will result in a drop of potential of whatever height 
the packet had in the outgoing buffer. Therefore, at the start of the active honest path, the buffers 
are full, while at the end of the path, a packet will be transferred to height zero (in the receiver's 
buffer). Intuitively, it therefore seems that tracking all packet movements along the active honest 
path should result in a drop of potential of at least 2n. As the counter-example in the footnote 
shows, this argument does not work exactly (we are only guaranteed a drop of n), but the structure 
of the proof is guided by this intuition. We begin with the following lemma. 

Lemma 6.14. Let C = N1N2 ■ ■ ■ Ni be a path consisting of I nodes, such that R = Ni and S ^ C. 
Suppose that in round t, all edges E{Ni, Ni-^-i) , 1 < i < I are active for the entire round. Let (j) 
denote the change in the network's non- duplicated potential caused by: 

1. (For 1 < i < I) Packet transfers across E{Ni, NiJ^i) in round t, 

2. (For 1 < i < I) Re-shuffling packets into Ni 's outgoing buffers during t, 

Then if Oni,N2 denotes Ni 's outgoing buffer along E{Ni, N2) and O denotes its height at the outset 
oft, we have: 

- If Oni,N2 has a flagged packet that has already been accepted by N2 before round t, then: 

<t)<-0 + l-l (2) 

- Otherwise, 

4)<-0 + l-2 (3) 

^'^An initial guess that the minimal potential drop equals "2n" for each blocked round is incorrect. Consider the 
case where the active path consists of all n — 2 intermediate nodes with the following current state: the first two 
nodes' buffers all have height 2n, the next pair's buffers all have height 2n — 1, and so forth, down to the last pair of 
internal nodes, whose buffers all have height n + 2. Then the drop in the network's non-duplicated potential is only 
n + 2 for this round. 



26 



Proof. The proof of this lemma is rather involved and relies heavily on the pseduo-code, so we have 
pushed its proof to Section [Tj where it is restated and proved as Lemma 17.151 ■ 

We can prove Lemma [6.131 Corollary. 

Proof of Lemma \6.13[ For every blocked round t, by the conforming assumption there exists a 
chain Ct connecting the sender and receiver that satisfies the hypothesis of Lemma [6. 141 Letting A^i 
denote the first node on this chain (not including the sender), the fact that the round was blocked 
means that A^i's incoming buffer was full, and then by Lemma [6?3l so was A'^i's outgoing buffer 
along E{Ni, N2). Since the length of the chain I is necessarily less than or equal to n. Lemma fG. 141 
says that the change in non-duplicated potential contributions of (j) (see notation there) satisfy: 

< -Oni,N2 + 1-1 < -2n + n-l < -n (4) 

Since (p only records some of the changes to non-duplicated potential, we use Statement 3 of Lemma 
16.111 to argue that the contributions not counted will only help the bound since they are strictly 
non-positive. Since we are not double counting anywhere, each blocked round will correspond to a 
drop in non-duplicated potential of at least — n, which then yields the lemma. ■ 

The following Lemma will bound the number of rounds that S needs to insert packets corre- 
sponding to the same codeword. 

Lemma 6.15. If at any time D — 2n^ distinct packets corresponding to some codeword bi have been 
inserted into the network, then R can necessarily decode message rui. 

Proof. Every packet that has been inserted into the network has either reached R or is in the 
incoming/outgoing buffer of an internal node (Claim [6771) . Since the maximum number of packets 
that are in the latter category is less than 4n^ (Claim [HTT]) . if D — 2n^ distinct packets corresponding 

to hi have been inserted, then R has necessarily received D — 6n^ = (1 — A) of these, and so 

by Fact 1 R can decode message mj. ■ 

We can now (restate and) prove the two main theorems of Section 14.31 

Theorem 14.21 Each message rrii takes at most 3D rounds to pass from the sender to the receiver. 
In particular, after 0{xD) rounds, R will have received at least 0{x) messages. Since each message 
has size M=^Pn^=0{n^) and D=^^=0{rfi) , after 0{x) rounds, R has received 0{x) bits of 
information, and thus our edge- scheduling adversarial protocol enjoys a linear throughput rate. 



Proof of Theorem Let t denote the round that S first tries to insert packets corresponding to a 
new codeword bi into the network. In each round between t and t -|- 3Z), either S is able to insert a 
packet or he isn't. By the pigeonhole principle, either D rounds pass in which S can insert a packet, 
or 2D rounds pass in which no packets are inserted. In the former case, R can decode by Lemma [6.151 
It remains to prove the theorem in the latter case. Lemma [6.13l says that the network non-duplicated 
potential drops by at least n in each of the 2D rounds in which no packets are inserted, a total drop 
of 2nD. Meanwhile, Lemma [6.111 guarantees that the increase to network potential between t and 
t -|- 3D caused by duplicated potential is at most by 2n^ — 8n? + 8n. Combining these two facts, 
we have that (not counting changes in potential caused by packet insertions) the network potential 
drops by at least 2nD — 2n^ + 8n^ — 8n between t and t -|- 3D. Since network potential can never be 
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negative, we must account for this (non-duplicated) potential drop with positive contributions to 
potential change. The potential already in the network at the start of t adds to the potential at most 
4n^— 14n^-|-8n^-|-8n (Claim [6^ . Therefore, packet insertions must account for the remaining change 
in potential of (2ni:'-2n3+8n2-8n)-(4n^-14n3+8n2+8n) = 2nD-An^+{l2n^ -16n) > 2nD-An^ 
(where the last inequality assumes n > 3). Lemma fG. Ill states that the only way network potential 
can increase (other than the contribution of packet duplication potential which has already been 
accounted for) is when S inserts a packet (a maximum increase of 2n per packet), so it must be 
that S inserted at least {2nD — 4n^)/2n = D — 2n^ packets into the network between t and t + 3Z), 
and again R can decode by Lemma fS. 151 ■ 

Theorem 14.31 The edge-scheduling protocol described in Section \4-S\ ( and formally in the pseudo- 
code of Section\B^ requires at most 0(n^ log n) hits of memory of the internal processors. 

Proof of Theorem |^.3t Packets have size log n to allow the packets to be indexed. Since each internal 
node needs to hold at most O(n^) packets at any time (it has 2(n — 2) buffers, each able to hold 2n 
packets), the theorem follows. ■ 



7 Edge- Scheduling Protocol: 

Pseudo-Code Intensive Claims and Proofs 

In this section we prove that our pseudo-code is consistent with the claimed properties that our 
protocol enjoys. 

The following lemma is the first attempt to link the pseudo-code with the high-level description 
of what our protocol is doing. Recall that a buffer is in normal (respectively problem) status 
whenever its status bit sb is zero (respectively one). Also, an outgoing buffer is said to have a 
flagged packet if Hpp ^ _L, and the flagged packet is the packet in the outgoing buffer at height 
Hpp. Notice that because the pseudo-code is written sequentially, things that conceptually happen 
simultaneously appear in the pseudo-code as occurring consecutively. In particular, when packets 
are moved between buffers, updating the buffers' contents and updating the height variables does 
not happen simultaneously in the code, which explains the wording of the first sentence in the 
following lemma. 

Lemma 7.1. At all times (i.e. all lines of code in Figures\^ 0, and\^ EXCEPT when packets 
travel between buffers ((^32-33), ^52-53), and ^89-90)), along any (directed) edge E{A, B) 
for any pair of internal nodes {A,B), we have that: 

1. If Hgp > HjN or Hop = ±, then Hgp = Hjn + 1 or Hgp = -L and IN[i] / _L V« € [L.-ff/jv] 
and = _L Vi e [Hin + l..2n]. 

2. If Hgp < Hjn, then \N\i] / _L Vi G [I. .Hgp - 1] and Vi G [Hgp + L.Hjn + 1], and 

= 1 Vi e [HjN + 2..2n] and IN[i?Gp] = -L. 

3. If Hpp > HouT, then sb = 1 and OUT[i] 7^ _L Vi G [L.Hout - 1] and OUT[Ffp] 7^ -L. 

4. If Hpp = ± or Hpp < Hour, then OUT[i] / _L Vi G [I..Hout]- 

5. The height o/IN, as defined by the number of packets (i.e. non-null entries) o/IN, is equal to 
the value of Hjn. 

6. The height 0/ OUT, as defined by the number of packets (i.e. non-null entries) of OUT, is 
equal to the value of Hqut- 
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7. Whenever ^53) is reached, Hgp G [1..2n] and Hjn G [0..2n - 1]. 

8. Whenever (El32j is reached, Hpp ^ 1. and Hqut £ [1..2n]. 

9. At all times (even those listed in the hypothesis above), Hfj\[, Hqut ^ [0..2n] and Hqp, Hpp G 
_L U [1..2n] (so the domains of these variables are well-defined). 

Additionally, during any call to Re-Shuffle: 

10. Whenever the conditional statement on line ^74) is satisfied, one packet will pass between 
buffers. In particular, there will be a buffer that was storing the packet before the call to Re- 
Shuffle that will not be storing (that instance of) the packet after the reshuffle. Similarly, there 
will be another buffer that has filled a vacant slot with (an instance of) the packet in question. 

11. Flagged packets do not move. More precisely, if Hpp ^ _L just before any call to Re-Shuffle, 
then Hpp and 0\JJ[Hpp] will not change during that call to Re-Shuffle. 

12. Either Hgp does not change during re-shuffling or Hgp has decreased to equal Hjn + 1. 
^4/50, if Hgp 7^ -L, then \N[Hgp] does not get filled at any point during re-shuffling. 

13. If HjN < 2n before Re-Shuffling, then Hj^ < 2n after Re-Shuffling. 

Proof of Lemma \ 7.1l We prove each Statement of the Lemma above simultaneously by using 
induction on the round and line number as follows. We first prove the Lemma holds at the outset 
of the protocol (base case). We then notice that the above variables only change their value in the 
lines excluded from the Lemma and lines ^35), ^38), ^46), jllSO), jllSS), (PST), ([6161-62), 
((6l64), and ((7191-94). In particular, we use the induction hypothesis to argue that as long as 
the statement of the Lemma is true going into each set of excluded lines and lines ([6l35), JHlSS), 
(I6l46), dSlSO), (I6l55), (PST), ([6161-62), ^64), and ([391-94), then it will remain true when the 
protocol leaves each of those lines. Using this technique, we now prove each Statement listed above. 

Base Case. At the outset of the protocol, Hgp and Hpp = _L, Hjn and Hqut = 0, and all entries 
of IN and OUT are _L ([3129-31 and [3133-35) so Statements 1-6 and 9 are true. 

Induction Step. We now prove that each of the above Statements hold after leaving lines ([6l32- 
33), ([6l35), ([6l38), ^46), ([6l50), ([1152-53), ([6l55), ^67), ([6161-62), ^64), ([389-90), and 
([391-94), provided they held upon entering these lines. 

Lines ([6132-33). The variables in Statements 1, 2, 5, 7, do not change in these lines, and hence 
these Statements remain valid by the induction hypothesis. Statement 3 is vacuously true, since 
Hpp is set to _L at the end of line ([6l33). Also, Statement 9 will remain valid as long as Statement 
8 does, as Hpp is set to _L on line ([6l33), and Hqut G [0..2n] would follow from Statement 8 
since upon entering these lines, Hqut G [1..2n] (Statement 8), and so subtracting 1 from H on line 
([6l33) ensures that Hqut will remain in [0..2n — 1] C [0..2n]. The first part of Statement 8, that 
Hpp / _L when ([6l32) is reached, follows immediately from Claim 17^ below together with the fact 
that ([6l30) must have been satisfied to reach ([6l32). 

We next prove Statement 6. Anytime lines ([6132-33) are reached, the decrease of one by 
Hqut on ([6l33) represents the fact that OUT should be deleting a packet on these lines. Since 
the induction hypothesis (applied to Statement 6) guarantees that Hqut matches the number of 
packets (non-bottom entries) of OUT before lines ([6132-33), the changes to Hqut and the height 
of OUT on these lines will exactly match/cancel provided OUT does actually decrease in height by 
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1 (i.e. provided 01}J[Hfp] ^ _L). Since Hpp is changed ([6133) a/fer deleting a packet ([6132), we 
may apply the induction hypothesis to Statements 3 and 4 to argue that OUJ[Hpp] 7^ _L as long 
as the value of Hpp was not _L when line ([6132) was reached. This was proven above for the first 
part of Statement 8. 

Statement 4 follows from the argument above as follows. Upon leaving ([61.33), Hpp = _L, so we 
must show OUT[i] 7^ _L Vi G [1..Hout]- As was argued above, Hpp ^ _L when ([61.32) is reached. If 
Hpp > Hqut when ([6l32) is reached, then by the induction hypothesis applied to Statement 3, on 
that same line OUT[i] / ± Vi G [I. .Hqut - 1] and 0\}J[Hpp] / _L. The packet at height Hpp will 
be deleted on ([6l32), so that OUT[i] 7^ _L Vi G [I. -Hqut - 1], but OUT[i] = _L for all i > Hour- 
Then when Hqut is reduced by one on ([6l33), we will have that OUT[z] 7^ ± Vi G [1..Hout], as 
required. 

If on the other hand Hpp < Hqut when ([6l32) is reached, then by the induction hypothesis 
applied to Statement 4, on that same line OUT[i] 7^ _L Vi G [1..Hout]- The packet at height Hpp 
will be deleted on ([6l32) and the packets on top of it shifted down one if necessary, so that after 
([6l32) but before ([6l33), we will have that OUT[i] 7^ _L Vi G [I. .Hqut - 1], but OUT[i] = _L for 
all i > Hqut- Then when Hqut is reduced by one on ([6l33), we will have that OUT[z] 7^ X Vz G 
[1..Hout], as required. 

The second part of Statement 8 also follows from the arguments above as follows. First, it was 
shown in the proof of Statement 6 that 0\JT[Hpp] 7^ _L when ([6l32) is reached. In particular, 
the height of OUT is at least one going into ([6l32), and then the induction hypothesis applied to 
Statement 6 implies that Hqut > 1 when ([6l32) is reached, and the induction hypothesis applied 
to Statement 9 implies that Hqut < 2n when ([6l32) is reached. 

Line ([6l35). Since only Hpp and OUT are modified on ([6l35), we need only verify Statements 3, 
4, 6, and 9 remain true after leaving ([6l35). Since Hpp is gets the value max{HouT, Hpp) on 
([6l35), Statement 9 will be true by the induction hypothesis (applied to Statement 9). Also, the 
height of OUT does not change, as ([6l35) only swaps the location of two packets already in OUT, 
so Statement 6 will remain true. 

Statement 3 is only relevant if Hpp > Hqut before reaching ([6l35), since otherwise Hpp = 
Hqut upon leaving ([6l35), and Statement 3 will be vacuously true. On the other hand, if Hpp > 
Hqut, then line ([6l35) is not reached since ([6l34) will be false. 

In order to reach ([61.35), Hpp 7^ ± on ([6l34), and so both Hqut and Hpp are not equal to _L 
when ([6l35) is entered (Claim [73]), and hence Hpp 7^ ± upon leaving ([6l35). Also, since ([6l35) 
is only reached if Hpp < Hqut ([61.34), we use the induction hypothesis (applied to Statement 4) 
to argue that before reaching ([6l35), we had that OUT[i] 7^ _L Vi G [1..Hout]- In particular, both 
OUJ[Hpp] and OUJ[Hout] are storing a packet, and the call to Elevate Flagged Packet simply 
swaps these packets, so that after the swap, it is still the case that OUT[z] 7^ _L Vi G [1..Hout]- 
Since in this case Hpp = Hqut after line ([61.35), Statement 4 will remain true. 

Line ([6l38). Hpp is the only relevant value changed on ([6l38), so it remains to prove the relevant 
parts of Statements 3, 4 and 9. We will show that whenever ([6l38) is reached, Hqut ^ [1..2n] and 
01}J[Hout] 7^ -L. If we can show these two things, we will be done, since when Hpp is set to Hqut 
on ([61.38), Statement 9 will be true. Statement 4 will follow from the induction hypothesis applied to 
either Statement 3 or 4, and Statement 3 will not be relevant. By the induction hypothesis (applied 
to Statement 9), Hqut G [0..2n] when ([6l38) is reached. The fact that ([6l38) was reached means 
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that the conditional statement on the Hne before (l6].37) was satisfied, and thus OUT is in normal 
status {sb = 0) and Hqut € [1..2n]. By the induction hypothesis (applied to Statement 3), the 
fact that s6 = going into ([61.37) implies that Hpp = _L or Hpp < Hqut going into ([6137), and 
then the induction hypothesis (applied to Statement 4) says that OUJIHout] 7^ -L when ([6138) is 
entered. 

Lines ([6l46) and ([6l50). The parts of Statements 1, 2, and 9 involving changes to Hgp are the 
only Statements that are affected by these lines. If the conditional statement on these lines are 
not satisfied, then no values change, and there is nothing to prove. We therefore consider the case 
that the conditional statement is satisfied. Then Hgp is set to Hjn + 1 on these lines, and hence 
Statement 2 is vacuously satisfied. Since we are assuming Hgp changes value on ([6146) or JBlSO), 
the conditional statement says that Hgp = -L or Hgp > Hjjy going into ([6l46) (respectively 
jniSO)). By the induction hypothesis (applied to Statement 1), IN[i] 7^ _L for all 1 < i < Hjjy, 
and IN[i] = _L for all i > Hj^. Therefore, since IN and Hj^ do not change on ([6146) or ((6l50), 
Statement 1 will remain true upon leaving these lines. Finally, for Statement 9, we need only show 
Hgp € [1..2n] upon leaving line ([6146) (respectively line JHlSO)). If Hgp > Hjn going into line 
([6146) (respectively line dfilSO)), then the change to Hgp is non-positive, and so the induction 
hypothesis applied to Statements 1 and 9 guarantee Hgp will be in [1..2n] upon leaving these lines. 
On the other hand, if Hgp = -L going into either of these lines, then Hjn < 2n, and the induction 
hypothesis applied to Statement 9 indicates that Hj]\f G [0..2n — 1] going into these lines, and hence 
Hgp € [1..2n] upon leaving either line. 

Lines ([6152-53). Notice that Hgp necessarily equals _L when leaving ([6l53), so Statement 2 above 
is vacuously satisfied. Also, neither Hqut, Hpp, nor OUT is modified in these lines, so Statements 
3, 4, 6, 8, and the parts of Statement 9 concerning these variables will remain valid by the induction 
hypothesis. 

We prove Statement 1 first. Recall that the height of an incoming buffer refers to the number of 
(non-ghost) packets the buffer currently holds. Since Hgp will necessarily equal _L when leaving line 
([6l53), we must show that IN[i] / _L Vi G [l.-Hjiy] and IN[i] = _L Vi € [Hij\/ + 1..2n] upon leaving 
line ([HI 53). Both of these follow immediately from the induction hypothesis applied to Statements 
1 and 2, as follows. By the induction hypothesis applied to Statements 1, 2, and 9, either Hgp = -L, 
1 < Hgp < HjN, or Hqp = Hjiy + 1 < 2n when line ([6l52) is reached. We consider each case: 

• If Hgp = HfN + 1 when we reach line ([6l52), then by the induction hypothesis (applied to 
Statement 1) it will also be true that 7^ ± Vi G [1..Hin] and IN[z] = _L Vz G [HiN + l-2n] 
when this line is reached. While on line ([6l53), first IN[iJG'p] = \h\[Hij^ + 1] is filled with a 
packet, and then Hjjsf is increased by one, and so Statement 1 will remain true by the end of 
line (053). 

• If 1 < Hgp < HjN when the protocol reaches ([6l52), then also when this line is reached we 
have that (by the induction hypothesis applied to Statement 2) \H[i] 7^ X Vi G [1..Hgp — 1] 
and yi G [Hgp + I-Hin + 1], and IN[i] = ± G [Hjn + 2..2n] and \N[Hgp] = -L. When a 
packet is inserted into slot Hgp and Hjn is increased by one on line ([6l53), we will therefore 
have that all slots between 1 and (the new value of) Hjiy will have a packet, and all other 
slots will be _L, and thus Statement 1 will hold. 

• If Hgp = -L going into line ([6l52), then Hgp will be set to Hjjy + 1 on this line, and then 
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we can repeat the argument of the top bullet point, provided Hj^ + 1 < 2n. If sbouT = 1, 
then Statement 4 of Lemma FT. 101 states that Hop 7^ -L when ((6l52) is reached, contradicting 
the fact we are in the case Hgp = -L. So we may assume sboijT = 0, and then the fact that 
([61,52) was reached means that ([6147) must have been satisfied because Hqut > Hjn- Since 
both of these variables live in [0..2n] by the induction hypothesis applied to Statement 9, we 
conclude Hjn < 2n on ([6147), and it cannot change value between then and ([6152). 

The first part of Statement 7 is proven in the above three bullet points. For the second part, if 
sbouT = when ([6l47) was evaluated earlier in the round, then the fact that ([6l53) was reached 
means Hqut > Hj^^ and then the second part of Statement 7 follows from the induction hypothesis 
applied to Statement 9. If on the other hand sbouT = 1 when ([6l47) was evaluated, then the second 
part of Statement 7 follows from Statement 5 of Lemma 17.101 

We now prove Statement 5. There are two relevant changes made on line JHlSS) that affect 
Statement 5: a packet is added to IN[i/Gp] and Hjp^ is increased by one. The argument in the 
preceding paragraph showed that when jfilSS) is reached, Hgp G [1..2n] and IN[i/Gp] = _L, and 
therefore the net effect of ([6153) is to increase the number of packets stored in IN by one and to 
increase Hjn by one. Therefore, since Statement 5 was true going into line JHlSS) by the induction 
hypothesis, it will remain true upon leaving jHlSS). 

It remains to prove the parts of Statement 9 not yet proven, namely that at all times Hjjy G 
[0..2n] and Hgp G _L U [1..2n]. As was proven in the third bullet point above, if ([6l52) is satisfied, 
then Hjiy < 2n, and hence the change there does not threaten the domain of Hgp- Also, ([6153) 
sets Hgp to _L, which is again in the valid domain. Meanwhile, on ((6l53) Hjn is changed to 
HjN + 1 < 2n, where the inequality follows from the induction hypothesis applied to Statement 7. 

Line ([6l55), ([6157), and ([6164). Since IN and Hgp are the only relevant quantities that change 
value on these lines, only the relevant parts of Statements 1, 2, and 9 must be proven. Since Hgp 
is set to _L on these lines. Statement 9 is immediate and Statement 2 is vacuously true. It remains 
to prove Statement 1. If Hgp = -L going into ((6l55), ([6157), or ((6l64), then Hgp and IN will not 
change, and the inductive hypothesis (applied to Statement 1) will ensure that Statement 1 will 
continue to be true upon exiting any of these lines. If 1 < Hgp < Hjn when JHlSS), ([6157), or 
([6164) is entered, then we may apply the induction hypothesis to Statement 2 to conclude that 
IN[i] / _L Vi G [1..Hgp - 1] and Vi G [Hgp + I-.Hin + 1], and IN[i] = 1 Vf G [Hin + 2..2n] and 
IN[i/G'p] = _L. In particular, there is a gap in IN storing a "ghost packet," and this gap will be filled 
when Fill Gap is called on ([61.55), ([6l57) or ([6l64). Namely, this will shift all the packets from 
height Hgp + 1 through H + 1 down one spot, so that after Fill Gap is called, \H[i] 7^ _L Vi G [l..Hij^] 
and IN[i] = _L Vi G [Hjn + l--2n], which is Statement 1. Finally, if Hgp > Hjn when ([6l55), ^57) 
or ([6l64) is entered, then Fill Gap will not do anything, and so IN will not change. Since Statement 
1 was true going into these lines (by our induction hypothesis), it will remain true upon exiting 
these lines. 

Line ([6161-62). The only relevant variables to change values on these lines are sbouT, Hqut, Hpp, 
and OUT, so we need only verify Statements 3, 4, 6, and 9 remain true after leaving ([6161-62). 
First note that Hpp 7^ ± upon reaching ([6l61) (since ([61.60) must be satisfied to reach ([6161-62)), 
so the induction hypothesis (applied to Statements 3 and 4) implies that OUT[iJpp] 7^ _L when 
([6l61) is reached. Therefore, Hqut > 1 when ([6l61) is reached, and hence Hqut G [1..2n] upon 
reaching ([6l61) by the induction hypothesis (applied to Statement 9). In particular, when Hqut is 
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reduced by one on ([61.62), we will have that Hqut £ [0..2n — 1] upon leaving ([61.62), as required. 
Also, Hpp will be set to _L upon leaving ([6l62), so Statement 9 remains true. 

Statement 6 also follows from the fact that 0\}'T[Hfp\ 7^ -L when ([6l61) is reached, as follows. 
Since (by induction) Statement 6 was true upon reaching ([6l61), the packet deleted from OUT on 
([6l61) is accounted for by the drop in Hqut on ([6l62). 

Statement 3 is vacuously true upon leaving ([6l62), so it remains to prove Statement 4. This 
argument is identical to the one used to prove Statement 4 in lines ([6132-33) above. 

Lines ([389-94). We first prove Statements 10-13, and then address Statements 1-9. We first prove 
Statement 10, i.e. that before Shuffle Packet is called on ([377), we have that Bf[M] ^ _L and 
BT[m+l] = 1. 

• If -Bi? is an outgoing buffer and Hpp = _L or Hpp < Hpp, then M = Hpp (the conditional 
statement on line ([380) will fail), and then Bp{M\ 7^ _L by the induction hypothesis applied 
to Statement 4. 

• \i Bp is an outgoing buffer and Hpp > Hpp, then M = Hpp — 1 (the conditional statement on 
line ([380) will pass), and then Bp[M] 7^ _L by the induction hypothesis applied to Statement 
3 or 4 (that M = Hpp — 1 is greater than zero follows from the fact that Hpp ^ _L implies 
FP^± (Claim [731), and then the induction hypothesis applied to Statement 6 says Hpp > 0). 

• If Bp is an incoming buffer and Bp[M + l] 7^ _L, then ([382) is satisfied and M is set to M + 1 
on line ([382), and then by construction Bp[M] / _L after line ([383). 

• Suppose Bp is an incoming buffer and Bp[M + 1] = -L. Notice that the induction hypothesis 
applied to Statement 2 and the fact that Bp[M + 1] = _L imply that Hop > Hpy = M. 
Therefore, the induction hypothesis applied to Statement 1 implies that Bp[M] 7^ _L. 

• If Bp is an outgoing buffer and Bplm] = _L, then the conditional statement on line ([384) 
will be satisfied, and hence m is set to m — 1. Thus after line ([385), Bplm + 1] = _L. 

• If Bp is an outgoing buffer and Bplm] 7^ _L, then the induction hypothesis applied to State- 
ments 3, 4, and 6 imply that Bplm + 1] = _L. 

• If Bp is an incoming buffer and Hgp = -L, then the value of m is not changed on line ([386), 
and so m + 1 = Hin + 1. The induction hypothesis applied to Statement 1 then implies that 
Bplm + l] = _L. 

• \i Bp is an incoming buffer and Hgp 7^ _L, then Bp[HiN + 2] = ± by the induction hypothesis 
applied to Statements 1 and 2, and thus after m is changed to m + 1 on ([387), we have that 
Bplm + 1] = Bp[HiN + 2] = ±, as required. 

For Statements 11-13, we need to change notation slightly, since Re-Shuffling can occur between 
two buffers of any types (except outgoing to incoming). To prove these statements, we therefore 
treat 4 cases: 1) Bp is an outgoing buffer, 2) Bp is an incoming buffer, 3) Bp is an outgoing buffer, 
4) Bp is an incoming buffer. We then prove the necessary Statements in each case. 

Case 1. The value of Bp[M] = OUT[M] is changed on line ([390), and hence Statement 11 
will hold provided M 7^ Hpp. The top two bullet points above guarantee that this is indeed 
the case. Statements 12 and 13 are not relevant unless Bp is an incoming buffer, which will 
be handled in case 4 below. 

Case 2. For Statement 13, the only relevant change to Hjn is on line ([392), where Hin 
decreases in value, and hence Statement 13 will remain true. For the first part of Statement 
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12, the only place Hop can change is line ([71.94). But if Hgp does change value here, then the 
conditional statement on the previous line guarantees that that Hgp decreases to Hjn + 1. 
Statement 11 and the second part of Statement 12 are not relevant to this case. 

Case 3. The value of BT[m + 1] = OUT[m + 1] is changed on line ([71.89), and hence 
Statement 11 will hold provided m + 1 / Hpp. But we have already shown Statement 10 
remains true, and in particular the slot that is filled on line ([7l89) was vacant. If Hpp ^ _L, 
then by the induction hypothesis applied to Statements 3 and 4, Ol}J[Hpp] ^ _L, and hence 
OUT[m + 1] = _L implies that m + 1 7^ Hpp. Statements 12 and 13 are not relevant to this 
case. 

Case 4. Since Bp is an incoming buffer, the condition on line ([3.74) implies that the value 
of m (which is the height of Bp) on hue ([373) must be at most 2n — 2 (M — m > 1 and 
M, m G [0..2n] by induction hypothesis applied to Statement 9). Therefore, when the height 
of Bp is increased by one on line ([391), it will be at most 2n — 1, and so Statement 13 will 
remain true. For the second part of Statement 12, we must show that the value of m + 1 on 
line ([389) is not equal to Hqp. In the case that Hqp 7^ _L on line ([386), the value of m will 
change to Hi^ + 1 on line ([387), and then the induction hypothesis applied to Statement 1 
implies that Hqp < Hjjy + 1 = m and so Hqp 7^ m + 1 on line ([389). Statement 11 and the 
first part of Statement 12 are not relevant for this case. 

It remains to verify Statements 1-9. There are two parts to proving Statements 1 and 2; we must 
show they hold when Bp is an incoming buffer and also when Bp is an incoming buffer. For the 
latter part. Statements 1 and 2 will be true if we can show that anytime an incoming buffer's slot 
is filled as on line ([389), the slot was either slot Hj^ + 1 (in the case that Hqp = -L) or Hjn^ + 2 
(in the case that Hqp 7^ -L). These facts follow immediately from the definition of m on line 
([373) and hues ([386-87) and ([389). For the former part. Statements 1 and 2 will remain true 
provided the packet taken from Bp on line ([389) is the top-most packet in Bp. Looking at the 
conditional statement on line ([382), if \N[Hjiy -|- 1] 7^ _L, then by the induction hypothesis applied 
to Statements 1 and 2, we must have that \N[Hjj^ + 1] is the top-most non-null packet, which is 
the packet that will be taken from Bp on line ([389) (since in this case M = Hjjy is changed to 
HiN + 1 on line ([383)). On the other hand, if \N[Hin -M] = -L on line ([382), then the induction 
hypothesis applied to statements 1 and 2 imply that IN [i?/Ar] is the top-most non-null packet, which 
is exactly the packet taken on line ([389) (since the conditional statement on line ([382) won't be 
satisfied, and hence the value of M won't be change on line ([383)). 

Similarly, there are two parts to proving Statements 3 and 4; we must show they hold when Bp 
is an outgoing buffer and also when Bp is an outgoing buffer. The former part will be true provided 
the packet taken from Bp on line ([379) is the top-most non-flagged packet. If Hpp = ±, then 
there is no flagged packet, and hence the packet taken from Bp should be the top packet, i.e. the 
packet in index Bp[Houp]- Investigating the definition of M on line ([372) and lines ([380-81) 
and ([389) shows that this will be the case if Hpp = _L. If Hpp 7^ _L and Hpp < Hqup, then 
investigating those same lines also shows the top packet will be taken from Bp (which is not flagged 
since Hpp < Hqup by assumption). If Hpp > Hqup, then line ([380) will be satisfied, shifting the 
value of M to Hqup — 1 on line ([381). By the induction hypothesis applied to Statement 3, this 
new value of M corresponds to the top-most non-flagged packet of Bp. The latter part will be true 
provided the packet given to Bp takes the first free slot in Bp (in particular, the packet will not 
over-write a flagged packet's spot). If Bp[Houp] 7^ -L on line ([384), then the induction hypothesis 
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applied to Statements 3, 4, and 6 imply that all slots of Bt between [1..Hout] are non-_L, and all 
spots above Hqut are _L. Therefore, (since in this case the conditional statement on line ([71,84) fails 
and hence the value of m does not change on the next line) the definition of m on line ([7l73) and 
line ([71,89) show that the first free slot of Bt will be filled. On the other hand, if Bt[Hout\ = -L on 
line (13,84), then by the induction hypothesis, we must have that Bt[Hout\ is the first free slot of 
Bt-, and by investigating lines ([373), ([384-85), and ([389), this is exactly the spot that is filled. 

Statements 5 and 6 remain true by the fact that Statement 10 was proven true and lines ([391) 
and ([392). To satisfy the condition on hne ([374), it must be that = M > 1 and Hbj, = m < 
2n, and hence the changes made to Hsp and Hbj. on lines ([391) and ([392) will guarantee the 
parts of Statement 9 regarding Hqut and Hjn remain true. Also, Hgp remains in the appropriate 
demain by induction applied to Statements 9, 12, and 13. Statements 7, 8, are not relevant. ■ 

Lemma 7.2. The domains of all of the variables in Figures\3\ and [7] are appropriate. In other 
words, the protocol never calls for more information to be stored in a node 's variable (buffer, packet, 
etc.) than the variable has room for. 

Proof. Below we fix a node N £ G and track changes to each of its variables. 

Outgoing Buffers OUT ^08). Each entry of OUT is initialized to _L on ^33). After this 
point. Statement 6 of Lemma 17.11 above guarantees OUT will need to hold at most Hqut 
packets, and since Hqut is always between and 2n (by Statement 9 of Lemma 17. ip and 
packets have size P, the domain for OUT is as indicated. 

Copy of Packet to be Sent p ([3l09). This is initialized to _L on ([3l34), and is only modified 
afterwards on ^38), ^33), and ^62). By Statements 3, 4, and 9 of LemmalLU OUT[H] / 
_L when p is set on ([6l38), and the changes on ([6l33) and ([6l62) reset p to _L. Therefore, the 
domain of p is as indicated. 

Outgoing Status Bit sb ([SllO). This is initialized to on ([3l35), and is only modified afterwards 
on lines ([6l33), ([6l28), and ([6l62), all of which change sb to or 1, as required. 

Packet Sent Bit d ^11). This is initialized to on ([3l35), and is only modified afterwards 
on lines ([6l26), ([6l40), and ([6l62), each of which change d to or 1, as required. 

Flagged Round Index FR ^12). This is initialized to _L on ([3l34), and is only modified 
afterwards on lines ([6l38), ([6l33), and ([6l62). The latter two lines reset FR to _L, while 
([6l38) sets FR to the index of the current stage and round t, and since there are 3D rounds 
per transmission and 2 stages per round ([5l02), so when FR is set to t on ([6l38), it will be 
in [0..6Z)], as required. 

Height of Outgoing Buffer H ^13). This is initialized to on ^36). After this point. 
Statement 9 of Lemma ITTTI above guarantees H € [0..2n], as required. 

Heigfit of Flagged Packet Hpp ^14). Statement 9 of Lemma 1711 guarantees that Hpp will 
lie in the appropriate domain at all times. 

Round Adjacent Node Last Received a Packet RR ^15). This is initialized to _L on ([3134), 
and is only modified afterwards when it is received on ([51,06), where it is either set to the 
received value or _L if nothing was received. As discussed below, the incoming buffer's value 
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for RR always lies in the appropriate domain domain, and hence so will the value received on 
(11106). 

Outgoing Buffer's Value for Adjacent Node's Incoming Buffer Height Hj]\i ([3ll6). This is ini- 
tialized to on JSlSS), and is only modified afterwards on line ([5106), where it is set to the 
value sent on ([5109) by the adjacent node, or _L in case no value was received. Since the value 
sent on ([5l09) will always be between and 2n (by Statement 9 of Lemma mil . Hjn has the 
required domain. 

Incoming Buffers IN ^18). Each entry of IN is initialized to _L on ([31.29). After this point. 
Statement 5 of Lemma [7m above guarantees IN will need to hold at most Hjn packets, and 
since Hjn is always between and 2n (by Statement 9 of Lemma mi) and packets have size 
P, the domain for IN is as indicated. 

Packet Just Received p ^19). This is initialized to _L on ([3l30), and is only modified after- 
wards on ((6l43), where it either is set to the value sent on ([6141) or _L in the case no value 
was received. Since the value sent on ([6141) has the appropriate domain (i.e. the size of a 
packet, P), in either case p has the appropriate domain. 

Incoming Status Bit sb ^20). This is initialized to on ([3l31), and is only modified afterwards 
on lines ^45), ((6149), ([6153), dSSS), (PST), and ^64), all of which change sb to or 1 
as required. 

Round Received Index RR ([31.21). This is initialized to —1 on ([3l31), and is only modified 
afterwards on lines ((6l53) and ([6164). The former sets RR to the index of the current stage 
and round t, and since there are 3D rounds per transmission and 2 stages per round ([5102), 
setting RR = t as on ([6l53) will put RR in [0..6D] as required. Meanwhile, ([6164) resets RR 
to —1. Thus, at all times RR E {0, 1}^^, as required. 

Height of Incoming Buffer H ^22). This is initialized to on ^31). After this point. 
Statement 9 of Lemma [TTTl above guarantees H € [0..2n], as required. 

Height of Ghost Packet Hgp ^23). Statement 9 of Lemma 17.11 guarantees that Hgp will lie 
in the appropriate domain at all times. 

Incoming Buffer's Value for Adjacent Node's Outgoing Buffer Height Hqut ^24). This is 
initialized to on ([3131), and is only modified afterwards on line ((5lll), where it is set to 
be one of the values sent on ([5l05) by the adjacent node, or _L in case no value was received. 
Since the value sent on ([51,05) (either Hqut or Hpp) will always be _L or a number between 1 
and 2n (see domain argument above for an outgoing buffer's height of flagged packet variable 
Hpp), Hqut has the required domain. 

Incoming Buffer's Value for Adjacent Node's Status Bit sbouT (13125). This is initialized to 
on ((3l31), and is only modified afterwards on lines (jSllO) and jSlll). Both changes assign 
sbouT to '0' or '1', as required. 

Incoming Buffer's Value for Adjacent Node's Flagged Round Index FR ([3l26). This is initialized 
to _L on ([3130), and is only modified afterwards on lines ([SllO-ll) and ((6l43). Each of these 
times, FR is either set to the value sent by the adjacent node, or _L in the case nothing was 
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received. Since the values sent on ([5].05) and ([6l45) live in [0..6-D] U _L (see argument above 
for an outgoing buffer's variable FR living in the appropriate domain), so does FR. 

Sender's Count of Packets Inserted k ^37). We want to argue that at all times, k corre- 
sponds to the number of packets (corresponding to the current codeword) that the sender 
has knowingly inserted. Lines ((4l39) and ([61.68) guarantee that k = at the outset of any 
transmission. The only other place k is modified is ([6l31) where it is incremented by one, 
so we must argue that ((6l31) is reached exactly once for every packet the sender knowingly 
inserts. By "knowingly" inserting a packet, we means that the sender has received verification 
that the adjacent node has received and stored the packet, and hence the sender can delete 
the packet. 

Suppose that in some round t, the sender sends a packet p as on ([6141). By Claim 17.71 
below, the sender will continue to try and send this packet to its neighbor until he receives 
confirmation of receipt. There are two things to show: 1) If the sender does not receive 
confirmation of receipt, then k is never incremented as on ([6131), and 2) If the sender does 
receive confirmation of receipt, then k is incremented exactly once. By "receiveing confirmation 
of receipt," we mean that line ([6130) is satisfied in some round t' when the sender's value for 
p equals the packet p sent in round t (see Definition 17.61 below) . Clearly, 1) will be true since 
([61.31) will never be reached if ([6l30) is never satisfied. For 2), suppose that in some later 
round t' > t the sender gets confirmation of receipt for p. Clearly line ([6l31) is reached this 
round, and k is incremented by one there. We must show k. will not be incremented due to 
p ever again. However, p will be deleted on line ([6l32) of round t', and therefore this packet 
can cause the sender to reach ([6l31) at most once. Thus, at all times k corresponds to the 
number of packets (corresponding to the current codeword) that the sender has knowingly 
inserted, as desired. Since each codeword has D packets, the domain for k is as required. 

Receiver's Storage Buffer Ir ([4l40). Each entry of Ir is initialized to _L on ([4l43), after which 
it is only modified on lines ([3,101) and ([6l66). The latter resets Ir, while the former sets 
entry k of Ir to the packet in /A^[l]. We show below that k will always accurately represent 
the number of current codeword packets the receiver has received, and hence will be a value 
between and D. It remains to show that /iV[l] will always hold a packet when ([3.101) 
is reached. We use Claim 17.31 below which states that for the receiver, anytime Hjjy > 0, 
Hgp = -L. Therefore, whenever ([399) is satisfied. Statement 1 of Lemma 17. II (together with 
the argument that IN has the appropriate domain) state that /A^[l] will hold a packet, as 
required. 

Receiver's Number of Packets Received k ([4l41). We want to show that k always equals the 
number of packets corresponding to the current codeword the receiver has received so far. 
Lines ([4l42) and ([6l66) guarantee that «; = at the outset of any transmission. The only 
other place k is modified is ([3l01) where it is incremented by one, so we must argue that 
([3l01) is reached exactly once for every packet (corresponding to the current codeword) that 
the receiver receives. By Statement 1 of Lemma 17. II and Claim [731 below, anytime ([3l01) is 
reached, IN[1] necessarily stores a packet. This packet is added to Ir on ([3l01) and then is 
promptly deleted from IN on ([3l02). By Claim [6?8l the receiver will never enter ([3lOO) twice 
due to the same packet, and hence ([3l01) is reached exactly once for every distinct packet 
corresponding to the current codeword (see comments on |3l00 and (3104). Therefore, k 
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always equals the number of packets corresponding to the current codeword the receiver has 
received so far, as desired. Since there are D packets per codeword, k, G [0..D], as required. ■ 

Claim 7.3. For any of the receiver's buffers IN, Hjjy = at the start of every round. Also, 
anytime Hjjy > 0, Hgp = -L- 

Proof. H = HiN is set to at the outset of the protocol ([31.31). The first statement follows 
immediately from line ((3102), where each of the receiver's incoming buffers IN have Hjn reset 
to zero during the re-shuffle phase of every round. For the second statement, we will show that 
whenever H changes value from in any round t, that Hqp will be set to _L at the same time, and 
neither will change value until the end of the round when H will be reset to zero during re-shuffling. 
In particular, the only place H can change from zero is on ([6153). Suppose ([61,53) is reached in 
some round t, changing H from zero to 1, and also changing Hgp to _L. Looking at the pseudo-code, 
neither H nor Hgp can change value until line ([7ll02), where H is reset to zero. Therefore, H can 
only be non-zero between lines ([6l53) and ([51.21) (when Receiver Re-Shuffle is called) of a given 
round, and at these times Hgp is always equal to _L. ■ 

Claim 7.4. Let OUT he any outgoing buffer, and Hpp, FR, and sb denote the height of it flagged 
packet, round the packet was flagged, and status bit, respectively (see (SllO, [3ll2, [3ll4jJ. Then 
Hpp = _L 44> FR = _L. ^4/50, anytime OUT has no flagged packets (i.e. Hpp = -i-), OUT has 
normal status (i.e. sb = 0). 

Proof. The first statement is true at the outset of the protocol ([3l34), so it will be enough to make 
sure that anytime Hpp or FR changes value from _L to non-_L (or vice- versa), the other one also 
changes. Examining the pseudo-code, these changes occur only on lines ([6l33), ([6l38), and ([6l62), 
where it is clear Hpp takes on a non-_L (respectively _L) value if and only if FR does. 

The second statement is true at the outset of the protocol ([3134-35). So it is enough to show: 
1) anytime Hpp is set to _L, sb is equal to zero, and 2) anytime sb changes to one, Hpp ^ _L. 
The former is true since anytime Hpp changes to _L, sb is set to zero on the same line (([6l33) and 
([6l62)), while the latter is true since sb only changes to one on ([6l28), which can only be reached 
if FR 7^ _L ([6l27), which by the first statement of this claim implies Hpp 7^ -L. ■ 

Claim 7.5. 

1. Anytime sbouT is equal to 1 when Create Flagged Packet is called on line |5ll5j, Hpp 7^ _L. 

2. Anytime Send Packet is called on line ^5^17), the flagged packet has height at least one (i.e. 
Hpp is at least one anytime Send Packet is called). 

Proof. We prove the 2"*^ statement by separating the proof into the following two cases. 

Case 1: sbouT = at the start of Stage 2. Since Send Packet is called, the conditional statement 
on line ([5ll6) was satisfied. Therefore, since we are in the case sbouT = on that line, then 
Hqut > HfN- Tracing Hjn backwards, it was received on line ([5l06) and represents the value 
of HjN that was sent on line ([51.09). Using the induction hypothesis applied to Statement 9 
of Lemma [TTTl Hjj^ > and hence the value of Hqut on ([51,16) must be at least one. Since 
Hqut and Hjjy cannot change between lines ([51,15) and ([5ll6) of any round, when Create 
Flagged Packet was called, it was still true that sbouT = and Hqut > Hjjy > 0. Therefore, 
line ([6l37) will be satisfied and ([6l38) will set Hpp = Hqut > 1 as required. 
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Case 2: sbouT = 1 at the start of Stage 2. Let t denote some round where sbouT = 1 at the 
start of Stage 2. Our strategy will be to find the most recent round that sbpuT switched from 
to 1, and argue that the value that Hpp acquired in that round has not changed. So let 
to + 1 denote the most recent round that sbouT had the value at any stage of the round. 
We argue that sbouT = 1 by the end of to + 1, and sbouT = at the start of Stage 2 of round 
to (the round before to + 1) as follows: 

— If sbouT equals by the end of round to + 1, then it will at the start of round to + 2, 
contradicting the choice of to + 1. 

— If sbouT = 1 at the start of Stage 2 of round to, then sbouT must have changed its value 
to sometime between Stage 2 of round to and the end of round to + 1 (since sbouT = 
at some point of round to + 1 by definition). This can only happen on line ([6l33) inside 
the Reset Outgoing Variables function of round to + 1 (this is the only place that sbouT 
can be set to zero). However, since sbouT cannot change between the time that Reset 
Outgoing Variables is called on line jSlOT) and the end of the round, it must be that 
sbouT was equal to zero at the start of round to + 2, contradicting the choice of to + 1. 

Now since sbouT = at the start of round to + 1 (it cannot change between Stage 2 of to 
and the start of to + 1), and sbouT = 1 by the end of mathttto + 1, it must have changed 
on line ([61,28) of round to + 1 (this is the only line that sets sbouT to 1). In particular, the 
conditional statements on lines ([61.25) and ([6l27) must have been satisfied, and so d was equal 
to 1 on line ([6l25) of round to + 1. Since d is reset to zero during Stage 1 of every round 
([6l26), it must be that d was switched from to 1 on line ([6l40) of round to (this is the only 
place d is set to one). Thus, we have that Send Packet was called on line ([317) of round to. 
We are now back in Case 1 above (but for round to instead of t), and thus Hpp was set to a 
value of at least 1 on line ([6l38) of round to. It remains to argue that Hpp does not decrease 
in value between round to and line ([3 17) of round t. But Hpp can only change value on 
lines ([6l33), ([6l35), and ([6l38). For round to, the former two of these lines have both passed 
when the latter is called (setting Hpp > 1 as in Case 1). Meanwhile, between to + 1 and t, 
we know that ([6l33) and ([6l38) cannot be reached, as this would imply the value of sbpuT is 
zero sometime after to + 1, contradicting the choice of to + 1. The only other place Hpp can 
change is ([6l35), which can only increase Hpp. Thus in any case, _L 7^ Hpp > 1 when Send 
Packet is called on ([3 17) of round t. 

The proof of the 1'^* statement follows from the proof given in Case 2 above. ■ 

Definition 7.6. We will say that an outgoing buffer gets confirmation of receipt for a packet p 
that it sent across its adjacent edge whenever line ([6l30) (alternatively line (|121 46) for the node- 
controlling + edge-controlling protocol of Sections ISl fTT]) is reached and satisfied and the packet 
subsequently deleted on ([6l32) (respectively (|12L 50)) is (a copy of) p. 

Claim 7.7. Suppose (an instance of) a packet p is accepted by node B in round t (using the 
definition of "accepted" from Definition \6. 5|) . Then: 

1. Let t' be the first round afte^^ t in which B attempts to send (a copy of) this packet across 
any outgoing edge. Then the corresponding outgoing buffer OUT of B will necessarily have 
normal status at the start of Stage 2 oft' . 

^*The Claim remains valid even if t' is a round in a different transmission than t. 
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2. If B fails to get confirmation of receipt for the packet in the following round (i.e. either RR 
is not received on ^06) of round t' + 1, or it is received but RR < FR), then OUT enters 
problem status as on 16^28) of round t' + 1. OUT will remain in problem status until the end 
of the transmission or until the round in which it gets confirmation of receipt (i.e. until RR 
is received as on |51.06j with RR > t'). 

3. From the time p is first flagged as on ^38) of round t' through the time B does get confir- 
mation of receipt (or through the end of the transmission, whichever comes first), B will not 
have any other flagged packets, i.e. p, Ol}J[Hpp] = p and FR = t' . 

Proof. We prove Statement 1 by contradiction. Let t' denote the first round after t in which B 
attempts to send (a copy of) p across an edge E{B,C), i.e. t' is the first round after t that Send 
Packet is called by i3's outgoing buffer OUT such that the p that appears on line ([6l38) of that 
round corresponds to p. For the sake of contradiction, assume that sbouT = 1 at the start of Stage 
2 of round t'. Since sbpuT cannot change between the start of Stage 2 and the time that Create 
Flagged Packet is called on line JSllS), we must have that sbouT = 1 on line ([61,37) of round t', and 
hence ([6138) is not reached that round. In particular, when Send Packet is called on line ([5].17) (as 
it must be by the fact that p was sent during round t'), the packet p that is sent (which is p) was set 
in some previous round. Let t denote the most recent round for which p was set to p as on ([61.38) 
(this is the only line which sets p). Then by assumption t < t', and OUT had normal status at the 
start of Stage 2 of round t (in order for ([6l38) to be reached). Since OUT had normal status at the 
start of Stage 2 of round t, but by assumption OUT had problem status at the start of Stage 2 of 
round t', let t denote the first round such that t < t < t' and such that OUT had problem status 
at the start of Stage 2 of t. Since the only place OUT switches status from normal to problem is 
on ([6l28), this line must have been reached in round t. In particular, this implies that ([6l25) was 
satisfied in round t, which in turn implies that Send Packet was called in round t — 1 (since d is 
re-set to zero at the end of Stage 1 of every round as on ([6l26)). But this is a contradiction, since 
t < t — 1 < t', and so p = p was sent in a round before t', contradicting the choice of t'. 

For Statement 2, since B sent p in round t' and OUT had normal status at the Start of Stage 
2 of this round, we have that Hqut > Hjn on line ([316) (so that Send Packet could be called). 
Since sbouT, Hour, and Hjn cannot change between ([5ll5) and ([5ll7) of any round, FR is set to 
t' on ([6l38) of round t'. Also, d = 1 after the call to Send Packet of round t' ([6l40). Notice that 
neither FR nor d can change value between the call to Create Flagged Packet in round t' and the 
call to Reset Outgoing Variables in the following round. Therefore, if B does not receive RR or if 
RR < FR = t' when Reset Outgoing Variables is called in round t' + 1, then ([6l25) and ([6l27) 
will be satisfied, and hence OUT will enter problem status in round t' + 1. That OUT remains 
in problem status until the end of the transmission or until the round in which RR is received on 
([5l06) with RR > t' now follows from the following subclaim. (Warning: the following subclaim 
switches notation. In particular, to apply the subclaim here, replace (t,to) of the subclaim with 
(t' + l,t').) 

Subclaim. Suppose that at the start of Stage 2 of some round t, an outgoing buffer OUT 
has problem status and _L ^ FR = to- Then OUT will remain in problem status until the 
end of the transmission or until the round in which RR is received on ([5l06) with RR > to. 

Proof. OUT will certainly return to normal status by the end of the transmission ([6l62), in 
which case there is nothing to show. So suppose that t' > t is such that OUT first returns to 
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normal status (in the same transmission as t) as on ([61.33) of round t'. In particular, lines 
([61,29) and ([6l30) were both satisfied, so OUT must have received RR on ([5l06) earlier in 
round t', with RR > FR. If the value of FR on line ([61.30) equals to, then the proof is 
complete. So we show by contradiction that this must be the case. 

Assume for the sake of contradiction that FR ^ to on line ([6l30) of round t'. Since FR was 
equal to to at the start of Stage 2 of round t by hypothesis, FR must have changed at some 
point between Stage 2 of round t and round t'. Notice that between these rounds, FR can 
only change values on lines ([6l33) and ([6l38). Let t" denote the first round between t and 
t' such that one of these two lines is reached. Note that t" > t, since ([6l33) already passed 
by the start of Stage 2 (which is when the subclaim asserts FR = to), and ([6l38) cannot 
be reached in round t since OUT has problem status when ([61.37) of round t is reached (by 
hypothesis). 

• Suppose FR is first changed from FR = to on ([6l33) of round t". First note that 
because ([6l33) is the first time FR changes its value from to, it must be the case that 
FR was still equal to to on ([6l30) earlier in round t". Also, since ([6l33) is reached in 
round t", OUT returns to normal status. Since t' was defined to be the first round after 
t for which this happens, we must have that t" > t'. But by construction t" < t', so 
we must have that t" = t'. However, this is a contradiction, because our assumption is 
that FR / to on line ([6l30) of round t' = t", but as noted in the second sentence of 
this paragraph, we are in the case that FR = to on line ([61.30) of round t". 

• Suppose FR is first changed from FR = to on ([6l38) of round t". Then ([6l37) must 
have been satisfied, and thus OUT had normal status when Create Flagged Packet was 
called in round t". Since OUT had problem status at the start of Stage 2 of round t (by 
hypothesis), the status must have switched to normal at some point between t and t", 
which can only happen on ([6l33). But if ([6l33) is reached, then FR will be set to _L on 
this line, which contradicts the fact that FR was first changed from FR = to on ([6l38) 
of round t". 

This completes the proof of the subclaim. 

For the third Statement, first note that 0\}'T[Hpp\ = |j as of line ([6l38) of round t'. This is the case 
since sbouT = on line ([5ll2) (by Statement 1 of this claim), and then the fact that Send Packet 
is called in round t' means Hqut > Hjjy on ([5ll6), and therefore since none of these values change 
between ([5ll2) and ([5ll6), ([6l37) will be satisfied in round t'. Therefore, we will track all changes 
to OUT and Hpp from Stage 2 of round t' through the time p is deleted from OUT as on ([6132-33) 
of some later rouncf^. and show that none of these changes will alter the fact that 0\JT[Hfp] = p. 
Notice that (before the end of the transmission) Hpp only changes value on lines ([6l33), ([6l35), 
and ([6l38); while OUT only changes values on lines ^32), ^35), and ([7189-90). Clearly the 
changes to each value on ([6l35) will preserve 0\JT[Hpp] = p, so it is enough to check the other 
changes. Notice that ([6l32) is reached if and only if ([6l33) is reached, which by Statement 2 of 
this claim does not happen until OUT gets confirmation of receipt that p was successfully received 
by S's neighbor, and therefore these changes also do not threaten the validity of Statement 3. The 
change to Hpp as on ([6l38) can only occur if ([6l37) is satisfied, i.e. only if OUT has normal status, 

^^Or through the end of the transmission, whichever occurs first. 
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and thus again Statement 2 of this claim says this cannot happen until OUT gets confirmation 
of receipt that p was successfully received by B's neighbor. Finally, lines ([7189-90) will preserve 
0\JJ[Hfp] =phy Statement 11 of LemmaO 

That FR = t' from ((61.38) of t' through the time B gets confirmation of receipt for p was proven 
in the subclaim above. Also, p can only change on ([6l33) or ([6138), which we already proved (in 
the proof of the subclaim above) are not reached. ■ 

Corollary 7.8. At any time, an outgoing buffer has at most one flagged packet. 

Proof. This follows immediately from Statement 3 of Lemma 17.71 ■ 

Claim 7.9. For any outgoing buffer OUT, if at any time its Flagged Round value FR is equal to 
t, then OUT necessarily called Send Packet on line ^17) of round t. 

Proof. Suppose that at some point in time, FR is set to t. Notice that the only place FR assumes 
non-_L values is on ([6138), and therefore line ([6137) must have been satisfied in round t. Since the 
values for sbouT, Hqut, and -ff/Ar cannot change between lines ([5115) and ([5ll6), the statement 
on ([5116) will also be satisfied in round t, and consequently Send Packet will be reached in t. ■ 

Lemma 7.10. Suppose that sbouT = 1 when line ^47) is reached in round t on an edge linking 
buffers OUT and IN. Further suppose that IN does receive the communication {p,FR) from OUT 
on line ^6^43) of t. Also, let to denote the round described by FR, let h denote the height of the 
packet in OUT in round tg, and let h! denote the height of IN at the start of round tg. Then the 
following are true: 

1. to is well-defined (i.e. FR ^ 1. and FR < tj. 

2. h> h'. 

3. OUT sent p to IN on line ^4\) of round Xq. Furthermore, the height of p in OUT when it is 
sent on line ^4\) of round t is greater than or equal to h. 

4. If the condition statement on line (ElSlJ of round t is satisfied, then the value of Hqp when 
this line is entered, which corresponds to the height in IN that p assumes when it is inserted, 
satisfies: _L 7^ Hgp <h' + l<2n. 

5. If the condition statement on line (ElSlJ of round t is satisfied, then Hjjy was less than 2n 
at the start of all rounds between to and t. 

Proof of Lemma PZ. 1 (A We make a series of Subclaims to prove the 5 statements of the Lemma. 

Subclaim 1. The value of FR that is sent on ([6l41) of round t is not _L. 

Proof. Since ([6141) is reached, Send Packet was called on ([5117). By Statement 2 of Claim 
17.51 we have that Hpp > 1 when Send Packet is called, and in particular Hpp 7^ _L on line 
(I5ll7). Since Hpp cannot change between ([5117) and ([6141), we have that Hpp 7^ _L on 
([6l41), and hence FRj^ ± on this line (Claim EH- 

Subclaim 2. to is well-defined (i.e. _L 7^ to < t). 

Proof. By the definition of to and Subclaim 1, to 7^ -L. Also, by looking at the three places 
that FR changes values (([6l33), ([6138), and ([6162)), it is clear that FR will always be less 
than or equal to the current round index. 
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Subclaim 3. t > to. 

Proof. That t > to is immediate {FR is reset to _L at the start of every transmission ([31.34) 
and ((61.62), after which time the FR can never attain a value bigger than the current round 
([6l38)). Therefore, we only have to show t 7^ to. For the sake of contradiction, suppose 
t = to. By hypothesis, sbouT = 1 when line ([6147) of round t = to is reached. Notice that 
sbouT is reset to on dUlO) of round t = to, so the only way it can be '1' on ([6147) later that 
round is if it is set to one on ((51.11). This can only happen if Hqut = -L or FR > RR. Since 
((6l47) is reached, ((6l44) must have failed, and since Hqut does not change values between 
the time it is received on ((Sll) and ((6l44), we have that Hqut 7^ -L on ((5lll). Therefore, 
we must have that FR > RR on ((5lll) of round t = to. 

Notice the value for FR here comes from the value sent by OUT on ((5l05), and this 
happens before line ((6l38) has been reached in round t = to. Therefore, the value of FR 
received on ((5lll) obeys FR < t = to (as noted above, FR can never attain a value bigger 
than the current round). Since RR < FR, line ((6l30) cannot have been satisfied since the 
time FR was set to its current value (within a transmission, the values RR assumes are strictly 
increasing, see ((3.34), ((6l53), and ((6l64)). Therefore, we may apply Claim [731 and Claim 
17.71 to argue that FR will not be changed on ((61.38) of round t = to (since OUT will have 
problem status), and consequently FR will still be strictly smaller than t = to when line 
((6l41) is reached of round to. This contradicts the definition of to as the value received on 
line ((1143) of round t. 

Subclaim 4. OUT had normal status at the start of Stage 2 of round to. For every round 
between Stage 2 of to + 1 through t — 1, OUT had problem status and FR = to. 

Proof. By definition of to, it equals the value of FR that was received in round t on line 
((6l43), which in turn corresponds to the value of FR that was sent on line ((6l41). Tracing 
the values of FR backwards, we see that the only time/place FR is set to a non-_L value (as 
we know it has by Subclaim 1) is on line ((6l38), and this must have happened in round to 
since FR = to by definition of to. Therefore, in round to, line ((6l38) must have been reached 
when Create Flagged Packet was called on line ((51.15); so in particular sbouT must have been 
zero on line ((6l37) to have entered the conditional statement. Since sbouT cannot change 
between the start of Stage 2 and line ((5ll5) (where Create Flagged Packet is called), it must 
have been zero at the start of Stage 2. This proves the first part of the subclaim. Now suppose 
there is a round t' between Stage 2 of to + 1 and t — 1 such that sbouT = at any time in 
that round (without loss of generality, let t' be the first such round). Since sbouT can only 
switch to zero on ((61.33) inside the call to Reset Outgoing Variables, it must be that this line 
is reached in t', and hence FR is also set to _L on this line. Since FR is only assigned non-_L 
values on ((6l38), FR can only assume values at least t' > to after this point. Thus, FR 
will not ever be able to return to the value of to, contradicting the fact that FR = to during 
round t. By the same reasoning, FR can never change value from to between the rounds to 
and t. 

Subclaim 5. OUT attempted to send p in round to. 

Proof. By definition, to denotes the value of FR during round t. Since FR can only be set to 
to on ((6l38) of round to, this line must have been reached in to. In particular, line ((6l37) was 
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satisfied during the call to Create Flagged Packet of round to, and hence sb = and H > Hjn 
at that time. Therefore, ((5ll6) will be satisfied when it is reached in round to, which implies 
Send Packet will be called on the following line. The fact that it was the same packet p that 
was sent in to as in t follows from Statement 3 of Lemma I7.7[ 

Subclaim 6. The height of p in OUT when it is transferred in round t is greater than or 
equal to h. 

Proof. Subclaim 5 stated that OUT attempted to send p in round to, and Subclaim 4 stated 
that OUT had normal status at the start of to. Therefore, the packet which was sent in round 
to (which is p) was initialized inside the call to Create Flagged Packet on line ([61.38). By 
observing the code there, we see that p is set to 0UT[i7], i.e. p has height H in round to, and 
Hpp is set to equal H on this same line. By Statement 3 of Claim [721 P = P will remain the 
flagged packet through the start of round t, and 0\JT[Hfp] = P- By Statement 11 of Lemma 
17.11 Hpp will not change during any call to re-shuffle. Indeed, since Subclaim 4 ensures that 
line ([6l38) is never reached from to + 1 through the start of t, the only place Hpp can change 
value is on ([6l33) or ([6l35). We know the former cannot happen between to + 1 and the start 
of t, since this would imply sbouT is re-set to zero on ([6l33) of that round, contradicting 
Subclaim 4. Therefore, Hpp can only change values between to + 1 and the start of t as on 
([6l35), which can only increase Hpp. Hence, from the time Hpp is set to equal the height of 
OUT in round to as on ([6l38) (which by deflnition is /i), Hpp can only increase through the 
start of round t. 

Subclaim 7. h> h' . 

Proof. This follows immediately from Subclaims 4 and 5 as follows. Because OUT tried 
to send the packet in round to (Subclaim 5) and because OUT had normal status in this 
round (Subclaim 4), it must be that the conditional statement on line ([51.16) of round to was 
satisfled, and in particular that the expression H > Hjn was true. Since h is deflned to be the 
value of ff, Hpp as of line ([6l38) of round to (Statement 6 of Lemma [7?T]) . this subclaim will 
follow if h' equals the value of -ff/Tv as of line ([6l38) of round to- But this is true by Statement 
5 of Lemma iTTl since the value of Hj^ on line ([51.16) comes from the value received on line 
([51.06), which in turn corresponds to the value of Hi^ sent on line ([51.09). 

Subclaim 8. If the conditional statement on line ([6l51) is satisfled in round t, then OUT's 
attempt to send p in round to failed (i.e. IN did not store p in to), and furthermore IN did 
not store p in any round between to and t. 

Proof. We prove this by contradiction. Suppose there is some round t € [to-.t — 1] in which 
IN stored p. This would mean that line ([6l51) was satisfled in round t, and in particular RR 
is set to t > to on ([6l53). But as already noted in the proof of Subclaim 2, for the remainder 
of the transmission, FR can never assume the value of a round before to- Similarly, once RR 
changes to t > to > FR on ([6l53) of round t, it can never assume a smaller (non-_L) value 
for the rest of the transmission {RR can only change to a non-_L value on line ([61.53)). But 
this contradicts the fact that RR < FR on ([6l51) of round t. 

Subclaim 9. If the conditional statement on line ([6l51) is satisfled in round t, then RR < to 
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between the start of to through Hne JHlSl) of round t. In particular, lines ([61.47) and ([61.51) 
will be satisfied for any round between to and t for which they are reached. 

Proof. RR is set to —1 at the start of any transmission (([3l31) and ([6l64)). Since the only 
other place RR changes value is ([6l53), it is always the case that the value of RR is less than 
or equal to the index of the current round. Thus, RR can only assume a value greater than 
(or equal to) to in a round after (or during) to. But this would mean there was some round 
between to and t — 1 (inclusive) such that ([6l53) was reached, which contradicts Subclaim 
8. The fact that ([6l51) will be satisfied whenever it is reached now follows immediately from 
Statement 3 of Claim [TTTI since in order to reach ([6l51), line ([6l48) must have failed, which 
means the communication on line ([6l43) was received. The fact that ([61.47) will be satisfied 
whenever it is reached follows from the fact that sbouT will always be set to one on ([5lll) of 
each round between to and t (the first part of this subclaim says RR < to, and Subclaim 4 
says that if FR is received on ([5lll), then FR = to). 

Subclaim 10. If the conditional statement on line ([6l51) is satisfied in round t, then there 
was no round between to + 1 and t — 1 (inclusive) in which IN received both Hqut and p. 

Proof. Suppose for the sake of contradiction that there is such a round, t. Notice that 
line ([6l51) of round t will necessarily be reached (since the conditional statement of line 
([6l44) will fail by assumption, ([6l47) will be satisfied by Subclaim 4, and ([6l48) will fail by 
assumption). However, line ([6l53) cannot be reached in round t (Subclaim 8 above), and 
therefore the conditional statement on line ([6l51) must fail. This contradicts Subclaim 9. 

Subclaim 11. If the conditional statement on line ([6l51) is satisfied in round t, then IN was 
in problem status at the end of round to, and remained in problem status until line ([6l53) of 
round t. 

Proof. We first show that shjN will be set to one on line ([6l45) or ([6l49) of round to. To 
see this, we note that if ([6l44) fails in round to, then necessarily ([61.47) and ([6l48) will both 
be satisfied. Afterall, ([6l47) is satisfied (Subclaim 7), and if ([6l48) failed, then ([6l51) would 
be reached and subsequently satisfied (Subclaim 9), which would contradict Subclaim 8. For 
every round between to + 1 and t, we will show that either the conditional statement on line 
([6l44) will be satisfied, or the conditional statements on lines ([6l47) and ([6l48) will both be 
satisfied, and hence shiN can never be reset to zero since lines ([6l53), ([6l55), and ([6l57) will 
never be reached. To see this, let t' G [to + l..t — 1]. If ([6l44) is satisfied for t', then we are 
done. So assume ([6l44) is not satisfied for t', and hence IN did not receive the communication 
on ([6l43) (Subclaim 10). This means ([6l48) will be satisfied. The fact that ([6l47) is also 
satisfied follows from Subclaim 9. 

Subclaim 12. If the conditional statement on line ([6l51) is satisfied in round t, then between 
the end of round to and the time Receive Packet is called in round t, we have that Hqp 7^ -L 
and Hgp <h' + l<2n. 

Proof. As in the proof of Subclaim 11, either line ([6l46) or ([6l50) will be reached in round 
to (since either line ([61.45) or ([6l49) is reached). The value of Hj^ at the start of round 
to is h' by definition. Since h' < h < 2n (the first inequality is Subclaim 7, the second is 
Statements 6 and 9 of Lemma [7T]l . and since Hjn cannot change value between the start of 
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to and the time Receive Packet is called, we have that the value of Hjn < 2n when either line 
([61,46) or ([61,50) is reached. Therefore, these lines guarantee that _L / Hgp < h' + 1 < 2n 
after these lines. After this, there are five places Hgp can change its value: ([6l46), ([6l50), 
([6l53), ([6l55), and ^57). As in the proof of Subclaim 11, lines 1^55) and ^57) will not 
be reached at any point between to and t, nor will line ([6l53) by Subclaim 8. The other two 
lines that change Hgp can only decrease it (but they cannot set Hgp to _L). 

Subclaim 13. If the condition statement on line ([6l51) of round t is satisfied, then the value 
of Hgp when this line is entered, which corresponds to the height in IN that p assumes when 
it is inserted, satisfies: Hqp ^ -L and Hgp < h' + 1 < 2n. 

Proof. This follows immediately from Subclaim 12 since p is inserted into IN at height Hgp 
([6l53). 

Subclaim 14. If the condition statement on line ([6l51) of round t is satisfied, then Hji^ was 
less than 2n at the start of all rounds between to and t. 

Proof. Subclaim 12 implies that h' < 2n (so -ff/Ar had height strictly smaller than 2n at the 
start of round to). Searching through the pseudo-code, we see that -ff/Ar is only modified 
on lines ([6l53), and during Re-Shuffling ([391-92). Between rounds to and t, line ([6l53) is 
never reached (Subclaim 8), and hence all changes to Hjn must come from Re-Shuffling. But 
because Him was less than 2n when it entered the Re-Shuffle phase in round to. Statement 
13 of Lemma ITTT] guarantees that Hj^ will still be less than 2n at the start of round t. 

All Statements of the Lemma have now been proven. ■ 

Claim 7.11. Every packet is inserted into one of the sender's outgoing buffers at some initial height. 
When (a copy of) the packet goes between any two buffers Bi / B2 (either across an edge or locally 
during re-shuffling), its height in B2 is less than or equal to the height it had in Bi. If Bi = B2, 
the statement remains true EXCEPT for on line ^35). 

Proof. We separate the proof into cases, based on the nature of the packet movement. The only 
times packets are accepted by a new buffer or re-shuffled within the same buffer occurs on lines 
([1132), ([1135), ([1153), ([6l55), ^57), ^61), ^64), ([389-90), and ([3101-102). Of these, 
([6l35) is excluded from the claim, and the packet movement on lines ([6l32), ([6l55), ([6l57), ([6l61), 
([6l64), and ([3101-102) are all clearly strictly downwards. It remains to consider lines ([6l53) and 
([389-90). 

Case 1: The packet moved during Re-Shuffling as on ([389-90). By investigating 
the code on these lines, we must show that m+l < M. This was certainly true as of line ([374), 
but we need to make sure this didn't change when Adjust Heights was called. The changes 
made to M and m on ([383) and ([385) will only serve to help the inequality m + l < M, so 
we need only argue the cases for when ([381) and/or ([387) is reached. Notice that if either 
line is reached, by ([374) we must have (before adjusting M and m) that M — m > 2, and 
therefore modifying only M = M — 1 or m = m + l won't threaten the inequality m + l < M. 
It remains to argue that both ([381) and ([387) cannot happen simultaneously (i.e. cannot 
both happen within the same call to Re-Shuffle). If both of these were to happen, then it 
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must be that during this call to Re-Shuffle, there was an outgoing buffer Bi that had height 
2 or more higher than an incoming buffer B2 (see lines ([7172-74) and ([7l80) and ((71.86)). 
We argue that this cannot ever happen. By Claim 16.31 at the end of the previous round, we 
had that the height of Bi was at most one bigger than the height of B2. During routing, B2 
can only get bigger and Bi can only get smaller (([6153) and ([6133) are the only places these 
heights change). Therefore, after Routing but before any Re-Shuffling, we have again that 
the height of Bi was at most one bigger than the height of B2. Therefore, in order for Bi 
to get at least 2 bigger than B2, either a packet must be shuffled into -Bi, or a packet must 
be shuffled out of B2, and this must happen when Bi is already one bigger than B2- But 
analyzing ([7l72) and ([3.73) shows that this can never happen. 

Case 2: The packet moved during Routing as on ([6l53). In order to reach ([6l53), 
the conditional statements on lines ([6l47), ([6l48), and ([6l51) all must be satisfied, so p / -L, 
RR < FR, and either sbouT = 1 or Hqut > H (or both). We investigate each case separately: 

Case A: shouT = 1 on line ([6147). Then Statements 2-4 of Lemma [7770] implv that 
the height of the packet in Bi is greater than or equal to the height it will be stored into 
in i?2, as desired. 

Case B: sbour = and Hqut > Hjn on line ([6147). For notational convenience, 
denote the current round (when the hypotheses of Case B hold) by t. First note that 
Statements 1 and 2 of Lemma 17. II imply that the height the packet assumes in B2 (Hgp) 
is less than or equal to Hjn + I. Meanwhile, since sbouT = (it is set on ((Slll) of round 
t), the value received for Hqut on ([Ull) is not _L, and the value for FR received on 
dUll) is either _L or satisfies FR < RR. Notice that the case FR < RR is not possible, 
since then ((6153) would not be reached (([6l51) would fail). Therefore, FR = _L but 
Hqut 7^ -L, and so B2 received the communication sent by Bi on ([5l05) of round t, 
which had the first of the two possible forms. In particular, Hpp = ± at the outset of t, 
and since Hpp cannot change between the start of a round a line ([6l38) of the previous 
round, we must have that ([6l37) failed in round t — 1. By this fact and Claim [731 ^1 
had normal status when ([5ll6) was reached in round t — 1, and this will not be able to 
change in the call to Reset Outgoing Variables of round t because d = ([6l25) (since 
d is reset to zero every round on ([6l26), it can only have non-zero values between line 
([6l40) of one round and line ([6l26) of the following round IF a packet was sent the earlier 
round. However, as already noted this did not happen, as the fact that OUT had normal 
status and yet ([6l37) failed in round t — 1 implies that ([5ll6) will also fail in round 
t — 1). Therefore, Bi has normal status when Create Flagged Packet is called in round t, 
and in particular, Hpp is set to Hqut on ([6l38), i.e. the flagged packet to be transferred 
during t has height Hqut in Bi. Putting this all together, the packet has height Hqut 
in Bi and assumes height Hqp in B2. But as argued above, Hqut > Hjn + 1 > Hqp, 
as desired. ■ 

Claim 7.12. Before End of Transmission Adjustments is called in any transmission T ([6l61j, 
any packet that was inserted into the network during transmission T is either in some buffer (perhaps 
as a flagged packet) or has been received by R. 

Proof. As packets travel between nodes, the sending node maintains a copy of the packet until it 
has obtained verification from the receiving node that the packet was accepted. This way, packets 
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that are lost due to edge failure are backed-up. This is the high-level idea of why the claim is true, 
we now go through rigorous detail. 

First notice that the statement only concerns packets corresponding to the current codeword 
transmission, and packets deleted as on ([6161) do not threaten the validity of the Claim. We consider 
a specific packet p that has been inserted into the network and show that p is never removed from 
a buffer B until another buffer B' has taken p from B. We do this by considering every line of code 
that a buffer could possible remove p, and argue that whenever this happens, p has necessarily been 
accepted from B by some other buffer B' . Notice that the only lines that a buffer could possibly 
remove p (before line jSei) of T is reached) are: ([6132), (PSS), and ([6189-90). 

Line ([6153). This line is handled by Lemma mi Statements 1 and 2, which say that whenever 
a slot of an incoming buffer is filled as on line ([6153), it fills an empty slot, and therefore cannot 
correspond to removing (over-writing) p. 

Lines ([7189-90). These lines are handled by Lemma mi Statement 10. 

Line ([6l32). This is the interesting case, where p is removed from an outgoing buffer after 
a packet transfer. We must show that any time p is removed here, it has been accepted by 
some incoming buffer B' . For notation, we will let t denote the round that p is deleted from 
B (i.e. when line ([6l32) is reached), and to denote the round that B first tried to send the 
packet to B' as on ([6l41). By Statement 3 of Claim [721 is the round that p was most 
recently set to p as on line ([6l38) (note that to < t). Since line ([6l32) was reached in round 
t, the conditional statements on lines ([6l29) and ([6l30) were satisfied, and so -L 7^ RR > FR 
when those lines were reached. By Statement 3 of Claim 17.71 FR will equal to when ([6l30) 
is satisfied. Since in any round t', the only non-_L value that RR can ever be set to is t' 
([6l53), and since RR > to = FR ([6l30), it must be that ([6l53) was reached in some round 
t' G [to,t]. In particular, B' stored a packet as on ([6l53) of round t', which by Statement 3 
of Claim 17.71 was necessarily p. ■ 

Claim 7.13. Not counting flagged packets, there is at most one copy of any packet in the network 
at any time (not including packets in the sender or receiver's buffers). Looking at all copies (flagged 
and un-flagged) of any given packet present in the network at any time, at most one copy of that 
packet will ever be accepted (as in Definition \6. 5]) by another node. 

Proof. For any packet p, let Np denote the copies oip (both flagged and not) present in the network 
(in an internal node's buffer) at a given time. We begin the proof via a sequence of observations: 

Observation 1. The only time Np can ever increase is on line ^6^53). 

Proof. The only way for Np to increase is if (a copy of) p is stored by a new buffer. Looking 
at the pseudo-code, the only place a buffer slot can be assigned a new copy of p is on lines 
([6l32), ([6l35), ([1153), (055), (057), (061), (064), and ^89). Of these, only (053) and 
([3.89) could possibly increase Np, as the others simply shift packets within a buffer and/or 
delete packets. In the latter case, Np does not change by Statement 10 of Lemma m^ 

Observation 2. Suppose A (including A = S) first sends a (copy of a) packet p to B as on 
^41) of round to . Then: 

(a) The copy of p in A's outgoing buffer along E(A,B) (for which there was a copy made 
and sent on ^41 ) of round to ) will never be transferred to any of A 's other buffers. 
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(b) The copy ofp will remain in A 's outgoing buffer along E(A, B) as a flagged packet until it 
is deleted either when A gets confirmation of receipt (see Definition \7.6\) in some round t 
1^32), or by the end of the transmission as on 1^61). In the latter case, define t := 3D 
(the last round of the transmission) for Statement (c) below. 

(c) Between tg and line HE^OT) of round t, B will accept (a copy of) p from A as on (El53j 
at most once. Furthermore, the copy ofp in A's buffer cannot move to any other buffer 
or generate any other copies other than the one (possibly) received by B as on ^6^53). 

Proof. Statement (a) follows from Statement 3 of Claim [72] and Statement 11 of Lemma 17. 11 
together with the fact that lines ([6132) and JHlGl, [6l69) imply that the relevant copy of A 
will be deleted when it does get confirmation of receipt as in Definition 17.61 (or the end of the 
transmission). By Statement 3 of Claim [721 this copy ofp will be (the unique) flagged packet 
in A's outgoing buffer to B until confirmation of receipt (or the end of the transmission), 
which proves Statement (b). For Statement (c), suppose that B accepts a copy of p as on 
([61.53) during some round t' G [to,t]. Then RR will be set to t' on ([6l53) of round t', and 
RR cannot obtain a smaller index until the next transmission ((6153). By Statement 3 of 
Claim [721 FR will remain equal to tg from line ([6l38) of round to through the time ([6l33) 
of round t is reached. Therefore, between t' > tg and line ([6l33) of round t, we have that 
FR = to < t' < RR, and hence line ([6l51) can never be satisfied during these times, which 
implies ([6l53) can never be reached again after t'. This proves the first part of Statement (c). 
The second part follows by looking at all possible places (copies of) packets can move or be 
created: ([6l32), ([6l35), ([Sl53), jllSS), ([6l57), ^61), ^64), and ([389-90). Of these, only 
([6l53) and ([3.89-90) threaten to move p or create a new copy of p. However, the first part 
of Observation 2(c) says that ([6l53) can happen at most once (and is accounted for), while 
Statement 11 of Lemma 17.11 rules out the case that the packet is re-shuffled as on ([389-90). 

Observation 3. No packet will ever be inserted (see Definition \6. 6\) into the network more 
than once. In particular, for any packet p, Np = until the sender inserts it (i.e. some node 
accepts the packet from the sender as on ^53)), at which point Np = 1. After this point, 
the only way Np can become larger than one is if ^6^53) is reached, where neither the sending 
node nor the receiving node is S or R. 

Proof. Since the packets of S are distributed to his outgoing buffers before being inserted into 
the network ([4l38), ([6l65), and ([6167-70), and since S never receives a packet he has already 
inserted {S has no incoming buffers ([3ll7)) nor shuffles packets between buffers (([5l22) and 
([395-96)), a given packet p can only be insereted along one edge adjacent to the sender. The 
fact the sender can insert at most one (copy of a) packet p along an adjacent edge now follows 
from Observation 2 above for A = S. This proves the first part of Observation 3. 

By Observation 1, the only place Np can increase is on ([6l53). Whenever this line is reached, 
the copy stored comes from the one received on ([6l43), which in turn was sent by another 
node on ([6l41). The copy sent on ([6l41) in turn can only be set on ([6l38) (perhaps in an 
earlier round), so in particular a copy of the packet must have already existed in an outgoing 
buffer of the sending node. This proves that when Np goes from zero to one, it can only 
happen when a packet is inserted for the first time by the sender. The rest of Observation 3 
now follows from Observation 1, the first part of Observation 3, the fact that copies reaching 
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R do not increase Np (by definition of Np), and the fact R never sends a copy of a packet 
([31,07) and S never accepts packets ([31.17). 

Define a copy of a packet p in the network to be dead if that copy will never leave the buffer it is 
currently in, nor will it ever generate any new copies. A copy of a packet that is not dead will be 
alive. 

Observation 4. // a (copy of a) packet is ever flagged and dead, it will forever remain both 
flagged and dead, until it is deleted. 

Proof. By definition of being "dead," once a (copy of a) packet becomes dead it can never 
become alive again. Also, copies of a packet that are flagged remain flagged until they are 
deleted by Observation 2(b). 

The Claim now follows immediately from the following subclaim: 

Subclaim. Fix any packet p that is ever inserted into the network. Then at any time, there 
is at most one alive copy of p in the network at any time. Also at any time, if there is one 
alive copy ofp, then all dead copies of p are flagged packets. If there are no alive copies, then 
there is at most one dead copy of p that is not a flagged packet. 

Proof. Before p is inserted into the network, Np = 0, and there is nothing to show. Suppose p 
is inserted into the network in round to, so that A'p = 1 by the end of the round (Observation 
3). Since Np = 1, the validity of the subclaim is not threatened. Also, if this packet is 
dead, then the proof is complete, as by Observation 3 and the definition of deadness, no other 
(copies) of p will ever be created, and hence the subclaim will forever be true for p. So suppose 
p is alive when it is inserted. We will show that a (copy of an) alive packet can create at most 
one new (copy of a) packet, and the instant it does so, the original copy is necessarily both 
flagged and dead (the new copy may be either alive or dead), from which the subclaim follows 
from Observation 4. So suppose an alive copy of p creates a new copy (increasing Np) of itself 
in round t. Notice that the only time new copies of any packet can be created is on ([6l53) 
(see e.g. proof of Observation 2). Fix notation, so that the alive copy of p was in node ^'s 
outgoing buffer to node B, and hence it was i3's corresponding buffer that entered ([6l53) 
in round t. The fact that the alive copy of p in A^s outgoing buffer is flagged and dead the 
instant B accepts it on ([6l53) of round t follows immediately from Observation 2. ■ 

Lemma 7.14. Suppose that in round t, B accepts (as in Definition \ 6. 5]) a packet from A. Let 
Oa,b denote A's outgoing buffer along E{A,B), and let O denote the height the packet had in Oa,b 
when Send Packet was called in round t ^E^IT). Also let Ib,a denote B's incoming buffer along 
E(A,B), and let I denote the height of Ib,a o,t the start oft. Then the change in non- duplicated 
potential caused by this packet transfer is less than or equal to: 

-O + I+l OR -O (ifB = R) (5) 

Furthermore, after the packet transfer but before re-shuffling, Ib,a will have height /+ 1. 

Proof. By deflnition, B accepts the packet in round t means that ([6l53) was reached by B's 
incoming buffer along E{A,B) in round t. Since the packet is stored at height Hqp ([6l53), -B's 
non-duplicated potential will increase by Hgp due to this packet transfer (if B = R, then by 
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definition of non-duplicated potential, packets in R do not contribute anything, so there will be 
no change). By Statements 1 and 2 of Lemma \7A[ Hop < I + ^, and hence B's increase in non- 
duplicated potential caused by the packet transfer is at most I + 1 (or zero in the case B = R). 
Also, since B had height I at the start of the round, and B accepts a packet on ((6l53) of round t, 
B will have I + 1 packets in / when the re-shuffling phase of round t begins, which is the second 
statement of the lemma. 

Meanwhile, the packet transferred along E{A, B) in round t still has a copy in Oa,b (until 
A receives confirmation of receipt from B, see Definition 17. 6p . but by definition of non-duplicated 
potential (see the paragraph between Claim WM and Lemma [6.11|) . this (flagged) packet will no 
longer count towards non-duplicated potential the instant B accepts it as on jfilSS) of round t. 
Therefore, yl's non-duplicated potential will drop by the value Hpp has when B accepts the packet 
on jHlSS) (Statement 3 of Claim [72}, which equals O since Hpp cannot change between the time 
Send Packet is called on JSll?) and the time the packet is accepted on ([6153). Therefore, counting 
only changes in non-duplicated potential due to the packet transfer, the change in potential is: 
—O + Hgp < —O + 1 + 1 (or —O in the case B = R), as desired. ■ 

We now re-state and prove Lemma [6.141 

Lemma 7.15. Let C = N1N2 ■ ■ ■ Ni be a path consisting of I nodes, such that R = Ni and S ^ C. 
Suppose that in round t, all edges E{Ni, Ni-^-i) , 1 < i < I are active for the entire round. Let (j) 
denote the change in the network's non- duplicated potential caused by: 

1. (For 1 < i < I) Packet transfers across E{Ni, NiJ^i) in round t, 

2. (For 1 < i < I) Re-shuffling packets into Ni 's outgoing buffers during t, 

Then if Oni,N2 denotes Ni 's outgoing buffer along E(Ni,N2) and O denotes its height at the start 
oft, we have: 

- //Oatj^atj ^'^^ ^ fl(i-99^d packet that has already been accepted by N2 before round t, then: 

<t)<-0 + l-l (6) 

- Otherwise, 

4) < -0 + 1-2 (7) 

Proof. (Induction on I). 

Base Case: 1 = 2. So C = NiR. 

Case 1: Oni,r had a flagged packet at the start of t that was already accepted by A^2- Our aim 
for this case is to prove jH) for Z = 2. If O < 2, then -O ^ - 1 > -1 2 - 1 = 0, and then JH) 
will be true by Statement 3 of Lemma lB.lli So assume O > 2. Since E{Ni,R) is active during 
t and R had already accepted the packet in some previous round t < t, we have that RR > t 
([61,53), and A^i will receive this value for RR in R's stage one communication ([5],06), ([5l09). 
By Statment 3 of Claim[L7l FR<i < RR, and thus lines ([1129-30) will be satisfied in round 
t, deleting the flagged packet on ([6l32) and setting sb = 0. When Create Flagged Packet is 
called on ([51.15), a new packet will be flagged, with Hpp = Hqut = — 1 and FR = t 
(since O > 2, there will be at least one packet left in Oni,r of height O — 1 > by Lemma 
I7.ip . Letting I denote the height of the receiver's incoming buffer along E{Ni,R), we have 
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that 1 = (Claim [731) • Therefore, Hqut > Hjjy, and so the flagged packet will be sent as on 
([5], 17). Since R will receive and store this packet (since the edge is active and RR < t = FR, 
hnes ^44) and ^48) will fail, while hnes ^47) and jSlSl) will be satisfied), we apply 
Lemma 17.141 to argue there will be a change in non-duplication potential that is less than or 
equal to — (O — 1), which is ([6|) (for 1 = 2). 

Case 2: Either On^^r has no flagged packet at the start oft, or if so, it has not yet been accepted 

by R. Our aim for this case is to prove ^ for 1 = 2. If O = 0, then —O + / — 2 = 0, and (jT]) is 
true by Statement 3 of Lemma fS. Ill So assume O > 1. Then necessarily a packet will be sent 
during round t (JSllG) is necessarily satisfied since by assumption E{Ni,R) is active during 
t , Hqut > 1 by Lemma 17.11 and Hijy = by Claim 17. 3|) . We first show that the height of 
the packet in On^^r that will be transferred in round t (which will be the value held by Hpp 
when Send Packet is called in round t) is greater than or equal to O (whether or not it was 
flagged before round t): 

• If Onx,R did not have any flagged packets at the outset of t, then Hpp = -L at the start 
of t, and so s6 = and FR = _L at the start of t by Claim 17.41 Since Hpp cannot 
change between the call to Send Packet in the previous round and the call to Reset 
Outgoing Variables in the current round. Statement 2 of Claim [731 implies no packet was 
sent the previous round, and hence d = at the start of t {d was necessarily zero as of 
([61.26) of round t — 1, and as argued did not change to '1' on ([61.40) later that round). 
Consequently, sh will remain zero from the start of t through the time Create Flagged 
Packet is called in round t, and because Hqut = O > = I = Hji^, ([61.38) will be 
reached in round t, setting Hpp to O. 

• Alternatively, if On-i_,R does have a flagged packet at the outset of t, we argue that it will 
have height at least O when Send Packet is called in round t as follows. Let to < t 
denote the round Oni,r first sent (a copy of) the packet to R. We first show that A^i 
will not get confirmation of receipt from R (as in Definition 17. 6p for the packet at any 
point between rounds to and t — 1 (inclusive). To see this, note that since we are Case 
2, R has not accepted the fiagged packet by the start of t. This means that at all times 
between to and the start of t, RR < t(0- Meanwhile, by Statement 3 of Lemma [7?7l 
FR = to and Hpp / _L at the start of t. Since these do not change values before Reset 
Outgoing Variables is called in round t, line ([6l34) guarantees that if Hpp < O, then 
line ([61,35) will be reached, and thus in either case Hpp > O after the call to Reset 
Outgoing Variables. 

Therefore, since R will necessarily receive and accept the flagged packet sent (by the same 
argument used in Case 1), we may apply Lemma r7. 141 to argue that cj) ^ —O, which is ([7|) (for 
I = 2). 

Induction Step. Assume the lemma is true for any chain of length less that or equal to Z — 1, and 
let C be a chain of length I {I > 2). Since we will be applying the induction hypothesis, we extend and 

^°By Statement 3 of Claim [7?7l the packet flagged in to is the only packet Oni,r can send to R between to + 1 and 
the time R receives this flagged packet. Since we know R has still not accepted this flagged packet by the outset of t, 
this means that between to and t — 1, RR cannot be changed as on ([6j53). Since RR begins each transmission equal 
to —1 (([3j31) and ([6j64)) and can only be changed after this on ([6l53), necessarily RR < to through the start of t. 
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change our notation as follows: Let ON^,Nj (respectively In.^Nj) denote the height of Nis outgoing 
(respectively incoming) buffer along edge E{Ni,Nj) at the start of round t (before, the notation 
referred to the buffer, now it will refer to the buffer's height). Notice that if Oni,N2 ^ In2,Ni, then: 

< -On2,Ns + (/ - 1) - 1 < -In2,n, +l-2< -OnuN2 + 1-2 (8) 

where the first inequality is from the induction hypothesis applied to the chain N2 ■ ■ ■ R, the second 
follows from Lemma 16.31 and the third follows from the fact we are assuming Oni,N2 ^ In2,Ni- 
Therefore, both ((H) and jT]) are satisfied. We may therefore assume in both cases below: 

Oni,N2 > In2,Ni (9) 

Case 1: Onx,N2 had a flagged packet at the start of t that was already accepted by N2. If 

Oni,N2 = In2,n-i_ + 1, then by the same string of inequalities as in ([8]), we would have 4> < 
—Oni,N2 + ^ — 1, which is ([6]). Therefore, it remains to consider the case: 

Oni,N2 ^ In2,Ni + 2 (10) 

By an analogous argument to the one made in the Base Case, a packet will be transfered and 
accepted across E{Ni,N2) in round t that will cause the non-duplicated potential to change 
by an amount less than or equal to: 

(-OiVi,iV2 +1) +%2,^l +1 (11) 

Also, when the receiving node A'2 accepts this packet as on JHlSS), the height of the corre- 
sponding buffer increases by one on this line. We emphasize this fact for use below: 

Fact: After the Routing Phase but before the call to Re-Shuffle in round t, A'^2's incoming 
buffer along E{Ni,N2) has height In2,Ni + 1- 

Meanwhile, we may apply the induction hypothesis to the chain C' := N2 ■ ■ ■ R, so that the 
change in non-duplicated potential due to contributions 1 and 2 (in the hypothesis of the 
Lemma) on C' is less than or equal to: 

(a) —On2,Ns + (^ ~ 1) ~ 1) if On2,N3 had a flagged packet at the start of t that was already 
accepted by N^^. 

(b) —On2,N3 + (^ — 1) — 2, otherwise. 

Adding these contributions to (fTTI) . we have that: 

(l> < i{-0NuN2 + 1) + ^iV2,7Vi + 1) + {-On2,N; + (/ - 1) - x) 

= {-On^,N2 + {-On2,Ns + lN2,m) + (2 - x), (12) 

where x = 1 or 2, depending on whether we are in case (a) or (b) above. By Lemma 16.31 
-0N2,N-i + In2,Ni is either or -1. If -On2,N3 + In2,Ni = -1, then i-ON2,N:i + In2,Ni) + 
(2 — x) < 0, regardless whether x = 1 or 2, and hence ([12]) implies ([6]). Also, if x = 2, then 
(— OAr2,Af3+-^A'2,A^i) + (2 — x) < (by Lemma [6?3ll . and hence hence (fT2]l implies jH]). It remains 
to consider the case x = 1 and —On2,N3 + In2,Ni = 0, in which case (fT2]) becomes: 

</>< (-07Vi,jV2 + /-!) + ! (13) 

In order to obtain jB]) from (fT3]) . we therefore need to account for a drop of at least one more to 
(p. We will obtain this by the second contribution to (/) (see statement of Lemma) by arguing: 
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(a) After the Routing Phase of round t but before the call to Re-Shuffiing, the fullest buffer 
of N2 has height OAr2,Ar3 + 1, and there is at least one incoming buffer of A''2 that has this 
height. In particular, during the call to Re-Shuffie in round t, the first buffer chosen to 
transfer a packet from will be an incoming buffer of height 0^2, N3 + 1- 

(b) After the Routing Phase of round t but before the call to Re-Shuffling, the emptiest 
buffer of A''2 has height On2,N3 — 1, and there is at least one outgoing buffer of A''2 that 
has this height. In particular, during the call to Re-Shuffle in round t, the first buffer 
chosen to transfer a packet to will be an outgoing buffer of height On2,N3 — 1- 

Notice that if I can show these two things, this case will be done, as during the first call to 
Re-Shuffle in round t, we will have M — m > {Om2,N3 + 1) — {On2,N3 — 1) > 2 (the call to 
Adjust Heights can only help this inequality since the selection process on ([3. 72- 73) and the 
two items above guarantee ([TlSO) and ([386) will both fail if reached), and consequently the 
re-shuffle on ([389-90) will cause a drop of at least one to (p. 

We first argue (a). As noted at the beginning of Case 1 of the Induction Step, Fact 1 
implies that there will exist an incoming buffer of the required height (since we are assuming 
On2,N3 = In2,Ni)- Also, at the start of t, since A''2 has an outgoing buffer of height On2,Nz 
(namely, the outgoing buffer along E{N2, N^)), Lemma lOl guarantees that all of A''2's incoming 
buffers have hieght at most On2,Nz at the start of t; and also that all of A''2's outgoing buffers 
have height at most On2,Nz + 1 at the start of t. During the Routing Phase but before the 
Re-Shuffle Phase oft, outgoing buffers cannot increase in height ([6l33) and incoming buffers 
cannot increase in height by more than one ([6l53). Therefore, after transferring packets but 
before Re-Shuffling in round t, the fullest buffer in A^2 has height at most On2,N3 + 1, and as 
already argued, at least one incoming buffer has this height. The last part of (a) is immediate 
from the selection rules in ([372). 

We now argue (b). Since x = 1, we are in the case the outgoing buffer along E{N2.,N^) had a 
flagged packet at the start of t that had already been accepted by A^3 in some round to < t. 
By a similar argument that was used in Case 1 of the Base Case, the outgoing buffer along 
E{N2,N3) will reach lines ([6132-33) in round t. In particular, the height of the outgoing 
buffer along E{N2, N3) will drop by one on ([61.33), and thus this buffer has height Om2,N3 — 1 
after the call to Reset Outgoing Variables. Since this height cannot change before the call to 
Re-Shuffle, this outgoing buffer has height 0^2, N3 — 1 after the Routing Phase (but before the 
call to Re-Shuffle) in round t. Also, 0^2, N3 — 1 is a lower bound for the emptiest buffer in A''2 
just before the call to Re-Shuffle in round t, argued as follows. At the start of t, since has 
an incoming buffer of height In2,Ni = 0^2, N3 (namely, the incoming buffer along E{Ni, N2)), 
Lemma [6?3l guarantees that all of A''2's incoming buffers have hieght at least On2,N3 — 1 at the 
start of t; and also that all of N2^s outgoing buffers have hieght at least On2,N3 at the start of 
t. During the Routing Phase but before the Re-Shuffle Phase of t, incoming buffers cannot 
decrease in height ([6l53) and outgoing buffers can decrease in height by at most one ([6l33). 
Therefore, after transferring packets but before Re-Shuffling in round t, the emptiest buffer 
in N2 has height at least 0^2, N3 — 1, and as already argued, at least one outgoing buffer has 
this height. The last part of (b) is immediate from the selection rules in ([373). 

Case 2: Either Oni,N2 has no flagged packet at the start of t, or if so, it has not yet been 
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accepted by By the same argumenll^ used in Case 2 of the Base Case, there will be 
a packet transferred across E{Ni,N2) and accepted by A''2 in round t, and this packet will 
have height at least Oni,N2 in -^I's outgoing buffer. Therefore, by Lemma r7. 141 the change in 
non-duplicated potential due to this packet transfer is less than or equal to: 

— Oni,N2 + In2,Ni + 1 (14) 

Meanwhile, we may apply the induction hypothesis to the chain C := N2 ■ ■ ■ R, so that the 
change in non-duplicated potential due to contributions 1 and 2 (in the hypothesis of the 
Lemma) on C' is less than or equal to: 

(a) —On2,N3 + (^ — 1) — 1; if On2,N3 had a flagged packet at the start of t that was already 
accepted by N^^. 

(b) —On2,N3 + (^ — 1) — 2, otherwise. 

Adding these contributions to (fT4l) . we have that: 

(p < {-Oni,N2 + In2,Ni + 1) + i-ON2,N3 + (1-1) - x) 

= {-Oni,N2 + {-On2,n, + In2,Ni) + (2 - x), (15) 

where x = 1 or 2, depending on whether we are in case (a) or (b) above. Since the first term 
of (fTSl) matches jTl and the latter two terms match the latter two terms of (fT2]l . we follow 
the argument of Case 1 above to conclude the proof. ■ 

8 Routing Against a (Node-Controlling+Edge-Scheduling) Adversary 

8.1 Definitions and High-Level Description of the Protocol 

In this section, we define the variables that appear in the next section and describe how they 
will be used. 

As in the protocol for the edge-scheduling adversary model, the sender first converts the input 
stream of messages into codewords, and then transmits a single codeword at a time. The sender will 
allow (at most) AD rounds for this codeword to reach the receiver (for the edge-scheduling protocol, 
we only allowed 3D rounds; the extra D rounds will be motivated below). We will call each attempt 
to transfer a codeword a transmission, usually denoted by T. At the end of each transmission, 
the receiver will broadcast an end of transmission message, indicating whether it could successfully 
decode the codeword. In the case that the receiver cannot decode, we will say that the transmission 
failed, and otherwise the transmission was successful. 

As mentioned in Section 12.21 in the absence of a node-controlling adversary, the only difference 
between the present protocol and the one presented in Section |4] is that digital signatures are used 
to authenticate the sender's packets and also accompany packet transfers for later use to identify 
corrupt nodes. In the case a transmission fails, the sender will determine the reason for failure 
(cases 2-4 from Section [221 ^Iso F2-F4 below), and request nodes to return status reports that 

^^For the argument in the Base Case, we used the fact that the receiver's incoming buffer had height zero in 
order to conclude Hour > Hin (and thus a packet would be sent). Here, we use instead ^ to come to the same 
conclusion. 
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correspond to a particular piece of signed communication between each node and its neighbors. We 
will refer to status report packets as parcels to clarify discussion in distinguishing them from the 
codeword packets. 

We give now a brief description of how we handle transmission failures in each of the three cases: 

F2. The receiver could not decode, and the sender has inserted less than D packets 

F3. The receiver could not decode, the sender has inserted D packets, and the receiver has not 
received any duplicated packets corresponding to the current codeword 

F4. The receiver could not decode and cases F2 and F3 do not happen 

Below is a short description of the specific kind of information nodes will be required to sign and 
store when communicating with their neighbors, and how this information will be used to identify 
corrupt nodes in each case F2-F4. 

Case F2. Anytime a packet at height h in an outgoing buffer of A is transferred to an incoming 
buffer of B at height h' , A^s potential will drop by h and B's potential will increase by h' . 
So for directed edge E{A,B)^ A and B will each need to keep two values, the cumulative 
decrease in ^'s potential from packets leaving A, and the cumulative increase in -B's potential 
from those packets entering B. These quantities are updated every time a packet is transferred 
across the edge, along with a tag indicating the round index and a signature from the neighbor 
validating the quantities and round index. Loosely speaking, case F2 corresponds to packet 
duplication. If a corrupt node attempts to slow transmission by duplicating packets, that 
node will have introduced extra potential in the network that cannot be accounted for, and 
the signing of potential changes will allow us to identify such a node. 

Case F3. A and B will keep track of the net number of packets that have travelled across edge 
E{A, B). This number is updated anytime a packet is passed across the edge, and the updated 
quantity, tagged with the round index, is signed by both nodes, who need only store the most 
recent quantity. Loosely speaking, case F3 corresponds to packet deletion. In particular, the 
information signed here will be used to find a node who input more packets than it output, and 
such that the node's capacity to store packets in its buffers cannot account for the difference. 

Case F4. For each packet p corresponding to the current codeword, A and B will keep track 
of the net number of times the packet p has travelled across edge E{A,B). This quantity 
is updated every time p flows across the edge, and the updated quantity, tagged with the 
round index, is signed by both nodes, who for each packet p need only store the most recent 
quantity. We will show in Section [10] that whenever case F4 occurs, the receiver will have 
necessarily received a duplicated packet (corresponding to the current codeword). Therefore, 
the information signed here will allow the sender to track this duplicated packet, looking for 
a node that outputted the packet more times than it inputted the packet. 

We will prove that whenever cases F2-F4 occur, if the sender has all of the relevant quantities 
specified above, then he will necessarily be able to identify a corrupt node. Notice that each case 
of failure requires each node to transfer back only one signed quantity for each of its edges, and so 
the sender only needs n status report packets from each node. 

We will show that the maximum number of failed transmissions that can occur before a corrupt 
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node is necessarily identified and eliminated is reo Because there are fewer than n nodes that can 
be corrupted by the adversary, the cumulative number of failed transmissions is bounded by n^. 
This provides us with our main theorem regarding efficiency, stated precisely in Theorem 18.11 in 
Section [Q] (note that the additive term that appears there comes from the failed transmissions). 

8.2 Detailed Description of the Node-Controlling + Edge-Scheduling Protocol 

In this section we give a more thorough description of our routing protocol for the node- 
controlling adversary model. Formal pseudo-code can be found in Section [H 

Setup. As in the edge-scheduling protocol, the sender has a sequence of messages {mi,m2, . . . } 
that he will divide into codewords {61, 62, . . . }. However, we demand each message has size M' = 
^'^^^ A^^^" , so that codewords have size C" = and these are then divided into packets of size 
P' = P — 2k, which will allow packets to have enough roonJ^ to hold two signatures of size k. 
Notice that the number of packets per codeword is D = ^ = ^(p_2fc)A ~ ^1 which matches the 
value of D for the edge-scheduling protocol. One of the signatures that packets will carry with them 
comes from the sender, who will authenticate every packet by signing it, and the packets will carry 
this signature until they are removed from the network by the receiver. We re-emphasize Fact 1 
from the edge-scheduling protocol, which remains true with these new values: 

Fact 1'. If the receiver has obtained D — 6n^ = (1 — A) (^^^ distinct and un-altered packets 
from any codeword, he will be able to decode the codeword to obtain the corresponding 
message. 

The primary difference between the protocol we present here and that presented in Section [4] is the 
need to maintain and transmit information that will allow the sender to identify corrupt nodes. To 
this end, as part of the Setup, each node will have additional buffers: 

3. Signature Buffers. Each node has a signature buffer along each edge to keep track of (outgo- 
ing/incoming) information exchanged with its neighbor along that edge. The signature buffers 
will hold information corresponding to changes in the following values for a single transmis- 
sion. The following considers A^s signature buffer along directed edge E{A,B): 1) The net 
number of packets passed across E{A, B); 2) S's cumulative change in potential due to packet 
transfers across E{A,B). Additionally, for codeword packets corresponding to the current 
transmission only, the signature buffers will hold: 3) For each packet p that A has seen, the 
net number of times p has passed across E{A, B) during the current transmission. 

Each of the three items above, together with the current round index and transmission 
index, ahve been signed by B. Since only a single (value, signature) pair is required for items 
l)-2), while item 3) requires a (value, signature) pair for each packet, each signature buffer 
will need to hold at most 2 + D (value, signature) pairs. 



^^As mentioned in Section [2T2| the sender can eliminate a corrupt node as soon as he has received the status reports 
from every non-blacklisted node. The reason we require up to n transmissions to guarantee the identification of a 
corrupt node is that it may take this long for the sender to have the complete information he needs. 

^^The network is equipped with some minimal bandwidth, by which we mean the the number of bits that can 
be transferred by an edge in a single round. We will divide codewords into blocks of this size and denote the size 
by P, which therefore simultaneously denotes the size of any packet and also the network bandwidth. As in the 
edge-scheduling protocol, we assume P > 0{k + logn), so that P' is well-defined. 
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4. Broadcast Buffer. This is where nodes will temporarily store their neighbor's (and their own) 
state information that the sender will need to identify malicious activity. A node A's broadcast 
buffer will be able to hold the following (signed) information: 1) One end of transmission 
parcel (described below); 2) Up to n different start of transmission parcels (described below); 
3) A list of blacklisted nodes and the transmission they were blacklisted or removed from the 
blacklist; 4) For each B £ G, a list of nodes for which B has claimed knowledge of their status 
report; and 5) For each B & G (including B = A), A's broadcast buffer can hold the content 
of up to n slots of B's signature buffer. Additionally, the broadcast buffer will also keep track 
of the edges across which it has already passed broadcasted information. 

5. Data Buffer. This keeps a list of 1) All nodes that have been identified as corrupt and 

eliminated from the network by the sender; 2) Currently blacklisted nodes; and 3) Each node 
A G will keep track of all pairs (A^i,A^2) such that N2 is on the blacklist, and Ni claims to 
know A'^2's complete status report. 

The sender's buffers are the same as in the edge-scheduling protocol's Setup, with the addition of 
four more buffers: 

- Data Buffer. Stores all necessary information from the status reports, as well as additional 
information it will need to identify corrupt nodes. Specifically, the buffer is able to hold: 1) Up 
to n status report parcels from each node; 2) For up to n transmissions, the reason for failure, 
including the label of a duplicated packet (if relevant); 3) For up to n failed transmissions, a 
participating list of up to n nodes that were not on the blacklist for at least one round of the 
failed transmission; 4) A list of eliminated nodes and of blacklisted nodes and the transmission 
they were blacklisted; 5) The same as items l)-3) of an internal node's Signature Buffer; and 
6) The same as item 3) of an internal node's data buffer. 

- Broadcast Buffer. Holds up to 2n start of transmission parcels and the labels of up to n — 1 
nodes that should be removed from the blacklist. 

- Copy of Current Packets Buffer. Maintains a copy of all the packets that are being sent in 
the current transmission (to be used any time a transmission fails and needs to be repeated). 

The receiver's buffers are as in the Edge-Scheduling protocol's Setup, with the addition of a Broad- 
cast Buffer, Data Buffer, and Signature Buffers which are identical to those of an internal node. 

The rest of the Setup is as in the edge-scheduling model, with the added assumption that each 
node receives a private key from a trusted third party for signing, and each node receives public 
information that allows them to verify the signature of every other node in the network. 

Routing Phase. As in the edge-scheduling protocol, rounds consist of two stages followed by 
re-shuffling packets locally. The main difference between the two protocols will be the addition of 
signatures to all information, as well as the need to transmit the broadcast information, namely the 
status reports and start and end of transmission broadcasts, which inform the nodes of blacklisted 
and eliminated nodes and request status reports in the case a transmission fails. The two stages 
of a round are divided as they were for the edge-scheduling protocol, with the same treatment of 
routing codeword packets (with the addition of signatures). However, we will also require that each 
round allows all edges the opportunity to transmit broadcast information (e.g. status report parcels). 
Therefore, for every directed edge and every round of a transmission, there are four main packet 
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transfers, two in each direction: codeword packets, broadcast parcels, and signatures confirming 
receipt of each of these. The order of transmitting these is succinctly expressed below in Figure [8l 
At the start of any transmission T, the sender will determine which codeword is to be sent (the 
next one in the case the previous transmission was successful, or the same one again in the case the 
previous transmission failed). He will fill his Copy Of Current Packets Buffer with a copy of all of 
the codeword packets (to be used in case the transmission fails), and then fill his outgoing buffers 
with packets corresponding to this codeword. All codeword packets are signed by the sender, and 
these signatures will remain with the packets as they travel through the network to the receiver. 



Stage 
1 


A 


B 


Ha ■= Height of buffer along e{A,b) 
Height of flagged p. (if there is one) 
Round prev. packet was sent 
Confirmation of rec. of broadcast info. 


i?_B:=Height of buffer along E{A,B) 
— Round prev. packet was received 
Sig's on values for edge e{A,b) 




Send p. and Sig's on values for E{A,B) if: 


Receive packet if: 




."Enough" bdcst info has passed e(a,b), AND 


• "Enough" bdcst info has passed e(a,b) 


2 


.B is not on A's blacklist /eliminated, AND — 


AND 




- Ha> Hb or 


• A is not on i3's blacklist /eliminated 




— B didn't rec. prev. packet sent 








— Broadcast Information 



Figure 8: Description of Communication Exchange Along Directed Edge E{A, B) During the Routing 
Phase of Some Round. 

To compliment Figure [8] above, we provide a breif description of the information that should 
be passed across directed edge E{A, B) (B ^ S and A R, and A,B £ G, i.e. not eliminated) 
during some transmission T. The precise and complete description can be found in the pseudo- 
code of Section [9l We state once and for all that if a node ever receives inaccurate or mis-signed 
information, it will act as if no information was received at all (e.g. as if the edge had failed for that 
stage). 

Stage 1. A will send the same information to B as in the edge-scheduling protocol (height, 
height of flagged packet, round packet was flagged). Also, if A received a valid broadcast 
buffer from B in Stage 2 of the previous round, then A will send B confirmation of this fact. 
Also, if A knows that B has crucial broadcast information A needs, A specifies the type of 
broadcast information he wants from B. Meanwhile, A should receive the seven items that B 
signed and sent (see below), updating his internal variables as in the edge-scheduling protocol 
and updating its signature buffer, provided B has given a valij^ signature. 

At the other end, B will send the following seven items to A: 1) the transmission index; 
2) index of the current round; 3) current height; 4) index of the round B last received a 

^■^Here, "valid" means that A agrees with all the values sent by B, and B's signature is verified. 
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packet from A; 5) the net change in packet transfers so far across E{A, B) for the current 
transmission; 6) -B's cumulative increase in potential due to packet transfers across E{A, B) in 
the current transmission; and 7) if a packet p was sent and received in Stage 2, the net number 
of times p has been transferred across E{A, B) for the current transmission. Meanwhile, B 
will also receive the information A sent. 

Stage 2. A will send a packet to B under the same conditions as in the edge-scheduling 
protocol, with the additional conditions: 1) A has received the sender's start of transmission 
broadcast (see below), and this information has passed across E{A,B) or E{B,A); 2) A and 
B are not on (A's version of) the blacklist; 3) A does not have any end of transmission 
information not yet passed across E{A, B) or E{B,A); and 4) A does not have any changes 
to blacklist information yet to pass across E{A,B) or E{B,A). We emphasize these last two 
points: if A (including A = S) has any start of transmission, end of transmission, or changes 
to blacklist information in its broadcast buffer that it has not yet passed along edge E{A,B), 
then it will not send a packet along this edge. 

If A does send a packet, the "packet" A sends includes a signature on the following seven 
item^: 1) transmission index; 2) index of the current round; 3) the packet itself with sender's 
signature; 4) index of the round A first tried to send this packet to -B; 5) One plus the net 
change in packet transfers so far across E{A, B) for the current transmission; 6) ^'s cumula- 
tive decrease in potential due to packet transfers across E{A, B) in the current transmission 
including the potential drop due to the current packet being transferred; and 7) One plus the net 
number of times the packet currently being transferred has been transferred across E{A, B) 
for the current transmission. 

Also, A should receive broadcast information (if B has something in its broadcast buffer 
not yet passed along E{A,B)) and update its broadcast buffer as described by the Update 
Broadcast Buffer Rules below. 

At the other end, B will receive and store the packet sent by A as in the edge-scheduling 
protocol, updating his signature buffer appropriately, with the added conditions: 1) B has 
received the sender's start of transmission broadcast, and this information has been passed 
across E{A, B) or E{B, A); 2) The packet has a valid signature from S; 3) A and B are not on 
(-B's version of) the blacklist; 4) B does not have any end of transmission information not yet 
passed across E{A,B) or E{B,A); 5) B does not have any changes to blacklist information 
yet to pass across E{A,B) or E{B,A); and 6) The signatures on the seven items included 
with the packet from A is "valid. Additionally, if there is anything in S's broadcast buffer 
that has not been transferred along E{A,B) yet, then B will send one parcel of broadcast 
information chosen according to the priorities: 1) The receiver's end of transmission parcel; 
2) One of the sender's start of transmission parcels; 3) Changes to the blacklist or a node to 
permanently eliminate; 4) The identity of a node N on B^s blacklist for which B has complete 
knowledge of A^'s status report; 5) The most recent status report parcel A requested in Stage 
1 of an earlier round; and 6) Arbitrary status report parcels. 

^^Recall that packets have room to hold two signatures. The first will be the sender's signature that accompanies 
the packet until the packet is removed by the receiver. The second signature is the one indicated here, and this 
signature will be replaced/overwritten by the sending node every time the packet is passed across an edge. 
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For any edge E{A, S) or E{R, B), only broadcast information is passed along these directed edges, 
and this is done as in the rules above, with the exceptions mentioned below in the Update Broadcast 
Buffer Rules for S and R. Additionally, any round in which the sender is unable to insert any 
packets, he will increase the number of blocked rounds in his data buffer. The sender will also keep 
track of the total number of packets inserted in the current transmission in his data buffer. 

Re-Shuffle Rules. The Re-Shuffle rules are exactly as in the edge-scheduling protocol, with the 
exception that node's record the changes in non-duplicated potential caused by re-shuffling packets 
locally. 

Update Broadcast BufTer Rules for Internal Nodes. Looking at Stage 2 of the Routing 
Rules above, we have that in every round and for every edge E{A,B), each node B will have 
the opportunity to send and receive exactly one parcel of broadcast information in addition to a 
single codeword packet. The order in which a node's broadcast buffer information is transmitted is 
described above in the Routing Rules. 

For any node A £ V \ {R,S}, we now describe the rules for updating their broadcast buffer 
when they receive broadcast information. Assume that A has received broadcast information along 
one of its edges E{A,B), and has verified that it has a valid signatme. We describe how A will 
update its Broadcast Buffer, depending on the nature of the new information: 

The received information is the receiver's end of transmission broadcast (see below). In this case, 
A will first make sure the transmission index is for the current transmission, and if so, the 
information is added to ^'s broadcast buffer, and edge E{A, B) is marked as having already 
transmitted this information. 

The received information is the sender's start of transmission broadcast (see below). This broad- 
cast consists of a single parcel containing information about the previous transmission, followed 
by up to 2n — 2 additional parcels (describing blacklisted/eliminated nodes and labels of up 
to n previous transmissions that have failed). When A receives a parcel from the start of 
transmission broadcast, if A does not already have it stored in its broadcast buffer, it will add 
it, and edge E(A,B) is marked as having already transmitted this information. Additionally, 
A will handle the parcels concerning blacklisted nodes as described below. Finally, when A 
has received every parcel in the start of transmission broadcast, it will also remove from its 
blacklist any node not implicated in this broadcast (i.e. this will count as "A receives infor- 
mation concerning a node to remove from the blacklist," see below), as well as clearing its 
signature buffers for the new transmission. 

The received information indicates a node N to eliminate. If the information is current, then A 
will add the new information to its broadcast buffer and mark edge E{A, B) as already having 
passed this information. If N is not already on A's list of eliminated nodes EN , then A will 
add N to EN (in its data buffer), clear all of its incoming and outgoing buffers, its signature 
buffers, and its broadcast buffer (with the exception of start of transmission parcels). 

The received information concerns a node A'^ to add to or remove from the blacklist. If the 

received information did not originate in the current transmission (as signed by the sender) 

or A has more recent blacklist information regarding A^, then A ignores the new information. 
Otherwise, the information is added to ^'s broadcast buffer, and edge E[A, B) is marked as 
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having already transmitted this information. Additionally, parcels that are now outdated in 
^'s broadcast buffer are deleted (such as A^'s status report parcels or a parcel indicating a 
node A'' had A^'s complete status report). 

If N = A, then A adds n of its own status report parcels to its broadcast buffer, choosing 
these n parcels based on information from the relevant start and end of transmission parcels. 
A also will add his own signature to each of these parcels, so that they each one will carry 
two signatures back to the sender (^'s signature and the relevant neighbor's signature). 

The received information indicates B has some node A^'s complete status report. If A^ is on 

^'s blacklist for the transmission B claims knowledge for, then A stores the fact that B has 
complete knowledge of A^ in its data buffer {A will later use this information when requesting 
a specific parcel from B). 

The received information pertains to some node A^'s status report the sender has requested. A will 

first make sure that A^ is on its version of the blacklist. If so, the information is added to ^'s 
broadcast buffer, and edge E{A, B) is marked as having already transmitted this information. 
If this completes A's knowledge of A^'s status report, A will add to its broadcast buffer the 
fact that it now knows all of A^'s missing status report. 

At the end of a transmission, all nodes will clear their broadcast buffers except parcels concerning a 
blacklisted node's status report. All nodes also clear their version of the blacklist (it will be restored 
at the beginning of the next transmission). 

Update Broadcast BufTer Rules for Sender and Receiver. The receiver has the same rules 
as internal nodes for updating its broadcast buffer, with the addition that when there are exactly 



n rounds left in any transmission^^!. R will add to its broadcast buffer a single (signed) parcel that 
indicates the transmission index, whether or not he could decode the current codeword, and the 
label of a duplicate packet he received (if there was one). We will refer to this as the receiver's end 
of transmission parcel. 

The rules for the sender updating its broadcast buffer are slightly more involved, as the sender 
will be the one determining which information it requires of each node, as well as managing the 
blacklisted and eliminated nodes. The below rules dictate how S will update his broadcast buffer 
and status buffer at the end of a transmission, or when the sender receives new (appropriately 
signed) broadcast information along E{S,B). 

A transmission T has just ended. Note that the sender will have necessarily received and stored 
the receiver's end of transmission broadcast by the end of the transmission (Lemma Ill.lQp . 
In the case that the transmission was successful, S will clear his outgoing buffers and Copy 
of Old Packets Buffer, then re-fill them with codewords corresponding to the next message. 
If EN denotes the eliminated nodes and denotes the nodes on the sender's blacklist at 
the end of this transmission, then the sender will set Ot+i = i\EN\,\Bi:\, F,0), where F 

^^We note that because there is always an active honest path between sender and receiver, and the receiver's final 
broadcast has top priority in terms of broadcast order, the sender will necessarily receive this broadcast by the end 
of the transmission. Alternatively, we could modify our protocol to add an extra n rounds to allow this broadcast to 
reach the sender. However, the exposition is easier without adding an extra n rounds, and we will show that wasting 
the final n rounds of a transmission by having the receiver determine if it can decode with n rounds still left is not 
important, as the n wasted rounds is insignificant compared to the 0{n^) rounds per transmission. 




62 



denotes the number of failed transmissions that have taken place since the last corrupt node 
was eliminated. Finally, the sender will delete the information in his data buffer concerning 
the number of packets inserted and the number of blocked rounds for the just completed 
transmission. 

In the case that the transmission failed, S will clear his outgoing buffers and then re-fill them 
using the Copy of Old Packets Buffer, while leaving the latter buffer unchanged. The sender 

will determine the reason the transmission failed (F2-F4), and add this fact along with the 
relevant information from his own signature buffers to the data buffer. For any node (not 
including S or R or eliminated nodes) not on the sender's blacklist, the sender will add the 
node to the participating list Vj in his data buffer, and then add each of these nodes to the 
blacklist, recording that T was the most recent transmission that the node was blacklisted. 
Also, the sender will set: 



For both a failed transmission and a successful transmission, the sender will sign and add to his 
broadcast buffer the following parcels, which will comprise the sender's Start of Transmission 
(SOT) broadcast: Ot+i, a list of eliminated and blacklisted nodes, and the reason for failure 
of each of the last (up to n — 1) failed transmissions since the last node was eliminated. Note 
that each parcel added to the broadcast buffer regarding a blacklisted node includes the index 
of the transmission for which the node was blacklisted, and all parcels added to the broadcast 
buffer include the index of the transmission about to start (as a timestamp) and are signed 
by S. Notice that the rules regarding priority of transferring broadcast information guarantee 
that Ox+i will be the first parcel of the SOT that is received, and because it reveals the 
number of blacklisted and eliminated nodes and the number of failed transmissions to expect, 
as soon as each node receives ^^t+i, they will know exactly how many more parcels remain in 
the SOT broadcast. Nodes will not be allowed to transfer any (codeword) packets until the 
50 T broadcast for the current transmission is received in its entirety. 

The sender receives the receiver's end of transmission parcel for the current transmission. The 

sender will store this parcel in its data buffer. 

The sender receives information along E{S,B) indicating B has some node A^'s complete status 

report. If is on the sender's blacklist for the transmission B claims knowledge for, then the 
sender stores the fact that B has complete knowledge of N in its data buffer (the sender will 
later use this information when requesting a specific parcel from B). 

The sender receives information along E{S,B) that pertains to some node N's status report that 
the sender has requested. The sender will first make sure that N is on its blacklist and that 
the parcel received contains the appropriate information, i.e. the sender checks its data buffer 
to see which transmission was added to the blacklist and the reason this transmission 
failed, and makes sure the status report parcel is from this transmission and contains the 
information corresponding to this reason for failure. If the parcel has faulty information that 
has been signed by A'', i.e. A'' sent back information that was not requested by the sender. 



f {\EN\,\Br\,F,p) 



if the transmission failed and p was included in 



i\EN\,\Bj\,F,l) 
[ {\EN\,\B-r\,F,2) 



the receiver's end of transmission parcel 
if the transmission failed and S inserted D packets 
otherwise 
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then N is eliminated from the network. Otherwise, the sender will add the information to its 
data buffer. If the information completes A'^'s missing status report, the sender updates his 
broadcast buffer indicating A^'s removal from the blacklist, including the index of the current 
transmission and his signature. 

The sender will then determine if he has enough information to eliminate a corrupt node 
N' . If so, N' will be added to his list of Eliminate Nodes, and his broadcast buffer and 
data buffer will be wiped completely {except for the list of eliminated nodes). Also, he will 
abandon the current transmission and begin a new one corresponding to the same codeword. 
In particular, he will clear his outgoing buffers and re-fill them with the codewords in his Copy 
of Old Packets Buffer, leaving the latter unchanged, and he will skip to the "A transmission 
has just ended" case above, setting the start of transmission parcel JIt+i = (1^-^1)0,0,0). 

8.3 Analysis of Our Node-Controlling + Edge-Scheduling Protocol 

We state our results concerning the correctness, throughput, and memory of our adversarial 
routing protocol, leaving the analysis and proofs to Section [TOl 

Theorem 8.1. Except for the at most transmissions that may fail due to malicious activity, our 
Routing Protocol enjoys linear throughput. More precisely, after x transmissions, the receiver has 
correctly outputted at least x — messages. If the number of transmissions x is quadratic in n or 
greater, than the failed transmissions due to adversarial behavior become asymptotically negligible. 
Since a transmission lasts O(n^) rounds and messages contain O(n^) bits, information is transferred 
through the network at a linear rate. 

Theorem 8.2. The memory required of each node is at most 0{n^{k + logn)). 

Proofs. See Section [TOl ■ 

9 Pseudo-Code for Node-Controlling + Edge- Scheduling Protocol 

We now modify the pseudo-code from our edge-scheduling adversarial protocol to pseudo-code 
for the (node-controlling -|- edge-scheduling) adversarial model. The two codes will be very similar, 
with differences emphasized by marking the line number in bold. The Re-Shuffle Rules will remain 
the same as in the edge-scheduling protocol, with the addition of line ([71,76) (see Figure (T]). 
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Setup 








DEFINITION OF VARIABLES: 




01 


n 


:= Number of nodes in G; 




02 


D 


■in'' . 
A - 




03 


T 


:= Transmission index; 




04 


t 


:= Stage/Round index; 




05 


k 


:= Security Parameter; 




06 


P 


■■— Capacity of edge= 0{k + logn) 




07 


for every N £ V \ S 




08 




BB e K +5n] X {0,1}^+"; 
DB 6 [l..n^] x {0,1}^; 


-^-^ BroctdCfist Buffer 


09 




7^7^ Data Buffer. Holds BL and EN below, and info, as on line 151 


10 




BL e [l..n - 1] x {0, 1}^; 
EN € [l..n-l] x{0, 1}^; 


## Blacklist 


11 




## List of Eliminated Nodes 


12 




57Gjv,iv €{0,1}°('°«"); 


## Holds change in potential due to local re-shufHing of packets 


13 


for every N G G 




14 




SK, {PK}? 


## Secret Key for signing, Public Keys to verify sig's of all nodes 


15 




for every outgoing edge E(N, B) 


G G B ^ S and N ^ R 


16 




OUT € [2n] X {0,1}^; 


7^7^ Outgoing Buffer able to hold 2?i packets 


17 




SIGn,b e[D + 3]x {0, l}0(i°g" 


, -ff--ff- oigiidiuie Duiiei iui cuiieui iiaiib., iiiueAeu ao luiiuwo. 

-ff--ff' O J [J- J — licL liU. Ui C(i//t/it CUUC W U 1 U \J b li clllbiei 1 cU clCI Ubb -Cj l^i V , y 

-fj--ff- iDi^\^\ — iiei ciiaiige in jd o pui. uue lo p. iiaiibieib dciubb i2j\^i\jIjj 

5^JG'[3]= net change in iV's pot. due to p. transfers across £(Ar, 
^Tt"- 5/Ct[j)]= net no. of times packet p transferred across E(^N, 


18 




pG{0,l}^Ui; 


-^-^ Copy of packet to be sent 


19 




s&G {0,1}; 


7?^7r ijiiaiub Dii 


20 




dG{0,l}; 


## Bit indicating if a packet was sent in prev. round 


21 




FR € [0..8L>] U _L; 


## Flagged Round (index of round N first tried to send p to B) 


22 




RR e [-1..8L>] U_L; 


## Round Received (index of round that N last rec. a p. from A) 


23 




H G [0..2n]; 


## Height of OUT. Also denoted Hqut when there's ambiguity 


24 




Hfp G [1..2n] U_L; 


## Height of Flagged Packet 


25 




HiN € [0..2n] U _L; 


Height of incoming buffer of B 


26 




for every outgoing edge E{N, B) € G, including B = S and N = R 


27 




bp G {0, 1}^; 
aG {0,1}^; 


## Broadcast Parcel received along this edge 


28 




## Broadcast Parcel request 


29 




Cbp € {0, 1}; 


## Verification bit of broadcast parcel receipt 


30 




for every incoming edge E{A, N) £ G,A ^ R and N ^ S 


31 




IN G [2n] X {0,1}^; 


## Incoming Buffer able to hold 2n packets 


32 




S'/GA.iv G [0 + 3] X {0,l}°(i°sn 
pG{0,ir U±; 


'; ## Signature Buffer, indexed as on line 17 


33 




## Packet just received 


34 




s6 G {0, 1}; 


## Status bit 


35 




J?7?G {0,1}®-°; 


## Round Received index 


36 




J/ G [0..2n]; 


## Height of IN. Also denoted -ff/iv when there's ambiguity 


37 




Hgp G [1..2n] U _L; 


## Height of Ghost Packet 


38 




Hour G [0..2n] U_L; 


## Height of outgoing buffer or height of Flagged Packet of A 


39 




sbouT G {0, 1}; 


## Status Bit of outgoing buffer of A 


40 




FR e {0,1}^^ U±; 


ij^ii- Flagged Round index (from adjacent outgoing buffer A) 


41 




for every incoming edge E(A, N) G G, including A = R and N = S 


42 




bp€{Q, iK; 


## Broadcast Parcel to send along this edge 


43 




Cbp e {0, 1}; 


## Verification bit of packet broadcast parcel receipt 



Figure 9: Pseudo-Code for Internal Nodes' Setup for the (Node-Controlling -|- Edge-Scheduling) 
Protocol 
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INITIALIZATION OF VARIABLES: 

44 for every N € G 

45 Receive Keys; ## Receive {PK}^ and SK from KEYGEN 

46 Initialize BB, DB, BL, EN, SIGn,n; ## Set SIGn,n = 0, set each entry of DB and BB to ± 

47 for every incoming edge N) £ G,A^ R and N =^ S 

48 Initialize IN, S/G; ## Set each entry in IN to ± and each entry of SIG to zero 

49 p,Hgp,FR = ±; 

50 sb,sbouT,Cp,H,HouT = 0; RR = —1; 

51 for every incoming edge iV) e G, including A = R and N = S 

52 6p = _L; C6p = 0; 

53 for every outgoing edge E{N, B) ^ G,B ^ S and N ^ R 

54 Initialize OUT, SIG; ## Set each entry in OUT to ± and each entry of SIG to zero 

54 p,Hfp,FR,RR = L; 

55 sb,d,H,HiN.(}; 

57 for every outgoing edge E{N, B) G G, including B = S and N = R 

58 6p, a = -L; Cbp = 0; 

Sender's Additional Setup 

DEFINITION OF ADDITIONAL VARIABLES FOR SENDER: 

59 M. := {rni.rn-z. . . .} — Input Stream of Messages; 

60 COPY <E [D] X {0, 1}^ := Copy of Packets for Current Codeword; 

61 BB e [3n] X {0, 1}^ ■- Broadcast Buffer; 

62 DB € [l..n^ + + n] x {0, 1}^ := Data Buffer, which includes: 

63 BL € [l..n] x {0, 1}^ := Blacklist; 

64 EN e [l..n] X {0, 1}^ := List of Eliminated Nodes; 

65 K G [0..-D] := Number of packets corresponding to current codeword the sender has knowingly inserted; 

66 fix G {0, := First parcel of Start of Transmission broadcast for transmission T; 

67 /3t G [0..4L)] := Number of rounds blocked in current transmission; 

68 F G [0..n — 1] := Number of failed transmissions since the last corrupt node was eliminated; 

69 Vt G {0, 1}" := Participating List for current transmission; 

INITIALIZATION OF SENDER'S VARIABLES: 

70 K = 0; 

71 0i,F = O; 

72 fii = (0,0,0,0); 

73 Initialize BB, DB, Pi; ## Set each entry of DB to ±, add Hi to BB, and set Vi = G 

74 Distribute Packets; 

Receiver's Additional Setup 

DEFINITION OF ADDITIONAL VARIABLES FOR RECEIVER: 

75 Ir G [D\ X ({0, 1}^ U _L) := Storage Buffer to hold packets corresponding to current codeword; 

76 KG [O-.D] := Number of packets received corresponding to current codeword; 

77 Gt G {0, ijOC^+iog"-) := End of Transmission broadcast for transmission T; 

INITIALIZATION OF RECEIVER'S VARIABLES: 

78 K = 0; 

79 Gi = _L; 

80 for every outgoing edge E{R, B) G G: 

81 bp, a = _L; 

82 Initialize Ir; ## Sets each element of Ir to ± 
End Setup 



Figure 10: Additional Setup Code for (Node-Controlling + Edge-Scheduling) Protocol 
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Transmission T 




01 for every N e G, N ^ EN: 




02 


for 


every t < 2 * (4_D) ## The factor of 2 is for the 2 stages per round 


03 


if t (mod 2) = then: ## STAGE 1 




04 




Update Broadcast Buffer One; 




05 




for every outgoing edge E{N, B)^G,N^R,B^S 




06 




if Hfp^ ±: send {H, _L, _L); else: send {H - 1, Hfp, FR); 




07 




receive Signed(T,t,HiN,RR,SIG[l],SIG[2],SIG\p]); ## S/G[3], 6*"- coord sent on Hne 11, is kept as S/Gp] 


08 




Verify Signature Two: 




09 




Reset Outgoing Variables; 




10 




for every incoming edge E{A,N) £ G, N S, A ^ R-ff-ff "p" on line 11 refers to last p. rec'd on e(A,n) 


11 




send Sign{T, t, H, RR, SIG[1], SIG[3], SIG\p]); ## If p was from an old codeword, send instead: 






## Sign(T, t, H, RR, SIG[l], SIG[3], ±) 




12 




sbouT = 0; FR= _L; 




13 




receive {H, _L, _L) or {H, Hfp, FR); ## If _ff = ± or fr>rr, set sbouT—l; and 








## Hout=Hfp; O.W. set Hout=H; sbouT 


=0; 


14 


e 


Ise if t (mod 2) = 1 then: ## STAGE 2 




15 




Send/Receive Broadcast Parcels; 




16 




for every outgoing edge E{N, B) e G,N ^ R,B ^ S 




17 




if Him 7^ -L then: 




18 




Create Flagged Packet; 




19 




if sb=l or {sb=0 and H > Hin) then: 




20 




Send Packet; 




21 




for every incoming edge E(A, N) e G, N ^ S, A R 




22 




Receive Packet; 




23 




if iV ^ {S, R} and has rec'd SOT broadcast for T then: Re-Shuffle; 




24 




else if N = R and N has rec'd SOT broadcast for T then: Receiver Re-Shuffle; 




25 




else if N = S then: 




26 




Sender Re-Shuffle; 




27 




if All (non-L) values S received on line 07 had Hin = '2n then: /3t = /3t + 1; 




28 




if t = 2(47) — n) and N = R, then: Send End of Transmission Parcel; 




29 




if t = 2(4D) and N = S then: Prepare Start of Transmission Broadcast; 




30 




if t = 2(4D) then: End of Transmission Adjustments; 




End Transmission T 




31 Okay to Send Packet 








' N does not have (fii.T) in BB 


OR 






N has (^1,1) with Ot = {\EN\, \Bt\,F,*), but has not yet rec'd \EN\ parcels as in line 200b, 








F parcels as in line 200c, or \Bt:\ parcels as in line 200d 


OR 


32 


if < 


has rec'd the complete 50r broadcast, but every parcel hasn't yet passed across E(N,E) 


OR 






N 01 B £ BL 


OR 






N has Gt € BB, but this has not passed across E{N, B) yet 


OR 






N has BL info, in BB (as on line 115, items 3 or 4) not yet passed across E{N,B) 




33 


Return False; 




34 


else: Return True; 




35 Okay to Receive Packet 








N does not have (fiiiT) in BB 


OR 






N has (fixjT) with Q.t = {\EN\, \Bt\,F,*), but has not yet rec'd \EN\ parcels as in line 200b, 








F parcels as in line 2G0c, or \Bi\ parcels as in line 200d 


OR 


36 


if . 


N has rec'd the complete 50 T broadcast, but every parcel hasn't yet passed across E{A,N) 


OR 






N OT A£BL 


OR 






N has Gt € BB, but this has not passed across E{A,N) yet 


OR 






N has BL info, in BB (as on line 115, items 3 or 4) not yet passed across E{A,N) 


OR 


37 


Return False; 




38 


else: Return True; 





Figure 11: Routing Rules for Transmission T, (Node-Controlling + Edge-Scheduling) Protocol 
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39 


Ti f Qff liii't'/irt^'n n i/fi'ni fihlfv 




40 






41 


if (i = 1 : 


ff-ff- sent a packet previous round 


42 


d = 0; 




43 


if Tii? = ± or ± ^ FJ? > iii? 


7^7^ Didn't receive conf. of packet receipt 


44 


s6 = 1- 




45 


1± J. LJ. L -f— —1— . 




46 


if 1 =^ FR < RR- 




47 


ifiV = S' then: K = K + 1; 




48 


For i = 1, 2,p: S/Gfi] — value rec'd on line 07; 




49 


SIG\3\ = SIG\3] + Hpp- 


dtdt N — S skin this line 


uu 




R.GinovG p from OUT, shifting down pcickcts on top 






of p (if necessary) and adjusting SIGn,n accordingly 


51 


FR,p, Hfp = ±; sh = 0; H = H -1; 






if 1 =^ RR ^ FR and 1 3^; Hjpo <: H- 

11 -i— ~f— JUL \ -I JL CIIIU —1— -f— 11 F }^ \ 11 . 


ff-ff- Qiti ilUb icccivc iiiuol icceniiy oeni pdCAcL 


uo 




-^-^ Swap packets in out[h] and oy)T[Hpp\; Set Hfp—H; 


54 


Create Flagged Packet 




55 


if s6 = and H > Hin: 


## Normal Status, will send top packet 




p = OUT[/f]; Hfp = H;FR = t; 




57 


Send Packet 




uo 


d=l; 




59 


if Okay to Send Packet then: 


## If p is from an old codeword, send instead: 


60 


send Sign{T,t,p,FR,SIG[l] + l,SIG[3]+HFP,SIG\p]+l)- ## Sign{T,t,p,FR,SIG[l],SIG[3] + Hfp,±) 


61 






62 


receive Sign{l, t-2,p, FR, SIG[1], SIG[2], SIG\p]); 


## 5/G[3], e*** coord, sent on line 60, is kept as SIG[2] 


63 


if Hour = J- or Okay to Receive Packet is false: 


##Didn't rec. A's ht. info, or BB info prevents p. transfer 


64 


sh = 1; 




65 


if Hgp > H or {Hop = ± and H < 2n): 




66 


Hop = H + 1; 




67 


else if sbouT = 1 or Hqut > H: 


## A packet should 've been sent 


68 


Verify Signature. One; 




69 


if ( Verify Signature One returns false or 


## Signature from A was not valid, or 




p = _L or p not properly signed by 5) then: 


## Packet wasn't rec'd. or wasn't signed by S 


70 


sb = 1; 




71 


if Hop > H or {Hgp = ± and < 2n): 




72 


HGP=H+h 




73 


else if RR < FR: 


## Packet was rec'd and should keep it 


74 


For i = 1, 2,p: SIG[i] = value rec'd on line 62; 




75 


SIG[3] = SIG[3]+Hgp; 


N = R, skip this line 


76 


if Hop = ±: Hgp = H + 1; 


## If no slot is saved for p, put it on top 


77 


\N[Hgp]=p; 




78 


sb = 0; H = H + 1; Hgp = ±; RR = t; 




79 


else: 


## Packet was rec'd, but already had it 


80 


sb = 0; Fill Gap; Hgp = -L; 


See comment about Fill Gap on line 82 below 


SI 


else: 


TtTt A packet should NOT have been sent 


82 


sb = 0; Fill Gap; Hgp = -L; 


Jiji If packets occupied slots above the Ghost 






## Packet, then Fill Gap will Slide them down one slot. 






## updating SIGn,n to reflect this shift, if necessary 


83 


Verify Signature One 




84 


if Signature is Valid and Values are correct 


## N verifies the values A sent on line 60 are consistent: 


85 


Return true; 


## Change in s/G[i] and SiG[p] is '1', change in SJGp] is 


86 


else: 


## at least Hgp, (T, t) is correct and p. has sender's sig 


87 


Return false; 


88 


Verify Signature Two 


## N verifies the values B sent on line 11 are consistent: 


89 


if Signature is NOT Valid or Values are NOT Correct:## Change in S/G[i] and SJG[p] is '1', change in S/G[2] 


90 


RR, Hi iv = -L ; 


## is at most Hfp, and T and t are correct 



Figure 12: Routing Rules for Transmission T, (Node-Controlling + Edge-Scheduling) Protocol (cont) 
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91 Send/Receive Broadcast Parcels 

92 for every outgoing edge E(N, B) £ G, including N = R, B = S 

93 receive bp; 

94 Update Broadcast Buffer Two; 

95 for every incoming edge E(A, N) £ G, including N = S, A = R 

96 Determine Broadcast Parcel to Send; 

97 send bp; 



98 Update Broadcast Buffer One 

99 for every outgoing edge E(N, B) £ G, including N = R, B = S 

100 iffep^-L then: 

101 send ctp; 

102 Broadcast Parcel to Request; 

103 send a; 

104 for every incoming edge E{A, N) £ G, including N = S, A = R 

105 receive Chp; receive a; 

106 if a 7^ _L then: Update Broadcast Buffer; ## Update to preferentially send a 

107 if Cj,p = 1 then: Update Broadcast Buffer; ## Update BB that bp crossed E(A, N) 

108 Cbp = 0; 



110 if _L 6p has valid sig. and 



109 Update Broadcast Buffer Two 

N has received full SOT broadcast for T OR 

bp is a valid 50r broadcast parcel rec'd in correct order (see 115 and 200) 
## Here, a "valid" signature means both from B and the from node bp originated from, and 
## a "valid" SOT parcel means that N has already received all SOT parcels that 
## should have arrived before bp, as indicated by the ordering of line 115, items 2a-2d 

111 ctp = 1; 

112 if N = S: Sender Update Broadcast Buffer; 

113 else: Internal Node and Receiver Update Broadcast Buffer; 



114 Determine Broadcast Parcel to Send 

115 Among all information in BB, choose some bp £ BB that has not passed along E{A,N) by priority: 

1) The receiver's end of transmission parcel Oi 

2) The sender's start of transmission (SOT) broadcast, in the order indicated on line 200: 
a) (fix, T) b) (iV £ EN, T) c) (T', Fi, T) d) (TV £ BL, T', T) 

3) (A'', 0, T) — label of a node to remove from the blacklist, see line 165 

4) (A'', A'^, T') = label of a node A'' on BL for which N has the complete status report for T', see line 155 

5) A status report parcel requested hy A as indicated by a (received on line 105) 

6) An arbitrary status report parcel of a node on A'^'s blacklist 

116 Broadcast Parcel to Request 

117 a = ±; 

118 if B is on A^'s blacklist and A'' is missing a status report from B: 

119 Set a to indicate B's label and an index of the parcel N is missing from B; 

120 else if DB indicates that B has complete status report for some node N on BL (see lines 150-151, 155): 

121 if A' is missing a status report of node A'': 

122 Set a to the label of the node N and the index of a status report parcel from N that A'^ is missing; 



Figure 13: Routing Rules for Transmission T, (Node-Controlling + Edge-Scheduling) Protocol (cont) 
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123 Internal Node and Receiver Update Broadcast Buffer 

## Below, a broadcast parcel bp is "Added' only if it is not already in BB. Also, view BB as being 
## indexed by each bp with n — 1 slots for each parcel to indicate which edges bp has already traversed. 
## Then when bp is removed from BB, the edge "markings" are removed as well. 

124 if bp = Ot is receiver's end of transmission parcel (for current transmission T, see line 179): 

125 Add bp to BB and mark edge E{N,B) as having passed this info.; 

126 else if bp = (fJuT) is the first parcel of the sender's start of transmission {SOT) broadcast (see line 200a): 

127 Add bp to BB, and mark edge E(N, B) as having passed this information; 

128 if SIt = (*, 0, *, *) : Clear all entries of SIG, and set SIGn,n — 0; 

129 else if bp= (N,!:) is from the SOT broadcast indicating a node to eliminate, as on line 200b: 

130 Add bp to BB and mark edge E{N, B) as having passed this info.; 

131 if iV ^ EN: ## N is just learning N is to be eliminated 

132 Add N to EN; 

133 Clear all incoming and outgoing buffers, clear all entries of SIG, and set SIGn,n = 0; 

134 Clear BB, EXCEPT for parcels from current 50r broadcast; Clear DB, EXCEPT for EN; 

135 else if bp= {T',Fi,T) is from the 50T broadcast indicating why a previous trans, failed, as on line 200c: 

136 Add bp to BB and mark edge E{N, B) as having passed this information; 

137 else if bp= {N,T',T) is from the 50T broadcast indicating a node to blacklist, as on line 200d: 

138 Add N to BL; Add bp to BB and mark edge E{N, B) as having passed this information; 

139 Remove outdated info, from BB and DB; _ 

## This includes for any trans. T" T' removing from DB all entries of form {B, N see line 115, item 4; 
## and removing from BB: 1) (N, N,T"), see line 115 item 4, and 2) Any status report parcel of A'' for T" 

140 if N = N has not already added its own status report info, corresponding to T' to BB: 

## The following reasons for failure come from SOT. See lines 190, 193, and 196-197 

## The information added in each case will be referred to as the node's status report for transmission T' 

141 if entries of SIGn,n and SIG correspond to a transmission T" 7^ T': Clear SIG and set SIGn,n = 0; 

142 if T' failed as in F2: For each incoming and outgoing edge, sign and add to BB: {SIG[2], SIG[3],T'); 

143 Also sign and add (SIGn,n, T') to BB (see line 12 of FigureE}; 

144 else if T' failed as in F3: For each incoming and outgoing edge, sign and add (S/Gfl],!') to BB; 

145 else if T' failed as in F4: For each incoming^and outgoing edge, sign and add {SIG[p],Y) to BB; 

146 if iV has received |SLt| SOT parcels of form (A^, T', T) : Clear all entries of SIG and set SIGn,n = 0; 

147 else if bp= (A,0,T) is from sender, indicating a node to remove from BL, as on line 165: 

148 Remove A'' from BL; Add bp to BB and mark edge E{N, B) as having passed this information; 

149 Remove outdated info, from BB and DB as on line 139 above; 

150 else if bp= (B,N,T') indicates B has a blacklisted node A^'s complete status report for trans. T': 

151 if (iV,T',T) is on A'''s blacklist: Add fact that B has iV's complete status report to DB; 

152 else if bp is a status report jjarcel for trans. T' of some node (A,T',T) on BL, see lines 140-145 and 200d: 

153 if bp has valid sig. Jrom iV and concerns correct info.: 

## N finds (TV, T',T) and (T',Fi,T) in BB (from SOT broadcast) and checks that bp concerns correct info. 

154 Add bp to BB, and mark edge E{N, B) as having passed this information; 

155 if bp completes A^'s knowledge of A'^'s missing status report for transmission T': Add (N, 7V,T') to BB; 

156 Sender Update Broadcast Buffer ## Below, a parcel bp is "Added' only if it is not in DB 

157 if bp = Ot is receiver's end of transmission parcel (for current transmission T): 

158 Add bp to DB; 

159 else if bp indicates B has a blacklisted node A'^'s complete status report for trans. T': 

160 if (TV, T', T) is on S's blacklist: Add {B, N, I'J to DB; 

161 else if bp is a status report parcel of some node N on the sender's blacklist (see lines 140-145): 

162 Add bp to DB; 

163 if bp contains faulty info, but has a valid sig. from N : ^ Eliminate N; 

## S checks DB for reason of failure and makes sure A'' has returned an appropriate value 

164 if bp conipletes the sender's knowledge of A'^'s missing status report from transmission T': 

165 Sign {N, 0, T) and add to BB; ## Indicates that N should be removed from blacklist 

166 Remove outdated info, from DB; Remove (A',T') from BL; ^ 

## "Outdated" refers to parcels as on 159-160 whose second entry is N 

167 if bp completes_sender's knowledge of all relevant status reports from some transmission: 

168 Eliminate N; ## S can eliminate a node. See pf. of Thm lS.ll for details 

Figure 14: Routing Rules for Transmission T, (Node-Controlling + Edge-Scheduling) Protocol (cont) 
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169 Eliminate N 

170 Add {N,l) to EN; 

171 Clear BB, DB (except for EN), and signature buffers; 

172 l3j,F = 0; 

173 Vt+1 = P\ EN; 

174 fii+i = (|£Af|, 0,0,0); 

175 Sign and Add fii+i to BB; 

176 for every iV G EN, Sign and ^dd (A^, T + 1) to BB; 

177 /faW until End of Transmission Adjustments is called;## S does not begin inserting p's until next trans., 

## and S ignores all instructions for T until line 30 

178 Send End of Transmission Parcel 

179 Add signed Gt = {b,p',T) to BB ## fe is a bit indicating if R could decode, p' is 

## the label of a packet R rec'd twice, or else ± 

180 Prepare Start of Transmission Broadcast 

181 ## Let 0T = (b,p',T) denote Sender's value obtained from Receiver's transmission above (as stored in DB) 

182 if b = 1 then: ## R was able to decode 

183 Clear each entry of signature buffers holding data corresponding to T; 

184 flj+i ^ {\EN\,\BL\,F,0); 

185 else if 6 = then: ## R was not able to decode: a failed transmission 

186 F = F+1; 

187 Set Vt^V\ (EN U BL) and add (Vi, T) to DB; 

188 For each iV £ Pt \ 5: Add (Af, T) to BL; ## (N, T) records the trans. N was added to BL 

189 Clear outgoing buffers; 

190 ifpV-L: ## R rec'd a duplicate packet 

191 Add (p', T) to DB; Add SIG[p'] to DB; ## Record that reason T failed was F4 

192 nj+i = {\EN\,\BL\,F,p'); 

193 else if k < D: ## S did not insert at least D packets 

194 Add (1,T) to DB; Add 5'7G'[2] and SIG[3] to DB; ## Record that reason T failed was F2 

195 Qt+i = {\EN\,\BL\,F,1); 

196 else: 

197 Add (2,T) to DB; Add S'/G[l] to DB; ## Record that reason T failed was F3 

198 Qj+i = {\EN\,\BL\,F,2); 

199 Clear BB and SIG[i] for each i = 1, 2,p; Remove Qt from DB; 

200 Sign and to BB: ## The Siarf of Transmission (SOT) broadcast 

a) (01+1,1+1) 

b) For each N € EN, add the parcel (iV, T+1) 

c) For each failed transmission T' since the last node was eliminated, add the parcel (T' , Fi,T+l) 
## Here, Fi is the reason trans. T' failed (F2, F3, or F4). See pf of Thm. [8lT]for details 

d) For each A'' € BL, add the parcel (A'', T', T+1), where T' indicates the trans. A' was last added to BL 

201 /3r = 0; 

202 End of Transmission Adjustments 

203 it N S : Clear Gt, BL, all parcels from SOT broadcast, and info, of form {N, 0, T) from BB; 

204 for every outgoing edge E{N, B), B £ G,N ^ R,B ^ S: 

205 ifBFp/l: 

206 OUT[Hpp\ — _L; Fill Gap; ## Remove any flagged packet p from OUT, shifting 

## down packets on top of p if necessary 

207 sb = 0; FR,Hfp,p^ ±; H = H ~1; 

208 for every incoming edge E{A, N), AeG, A^ R: 

209 Hgp = -L; sb^O; RR= -1; Fill Gap; 

210 if A^ = S then: Distribute Packets; 

211 if A' = i? then: k = 0; Clear Ir; ## Set each entry of Ir to ± 

212 Distribute Packets 

213 «: = 0; Hqut = 2n; ## Set height of each outgoing buffer to 2n 

214 Fill each outgoing buffer with codeword packets; 

## If T was successful, make new codeword p's, and fill out. buffers and COPY with these. 
## If T failed or a node was just eliminated, use codeword packets in copy to fill out. buffers. 



Figure 15: Routing Rules for Transmission T, (Node-Controlling + Edge-Scheduling) Protocol (cont) 
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10 Node-Controlling + Edge- Scheduling Protocol: 
Proofs of Theorems 



We restate and prove our two main theorems for the node-controlling adversary Routing Proto- 
col: 

Theorem 18.21 The memory required of each node is at most 0{n^{k + logn)). 

Proof. By looking at the domains of the variables in Figures [9] and [TOl it is clear that the Broadcast 
Buffer and Signature Buffer maintained by all nodes, and the Data Buffer of the sender and Storage 
Buffer of the receiver require the most memory. Each of these require 0{n^{k + logn)) bits of 
memory, but each node must maintain 0{n) signature buffers, which yields the memory bound of 
0(n^(/c + logn)). It remains to show that the domains described are accurate, i.e. that the protocol 
never calls for the nodes to store in more (or different) information than their domains allow. The 
proof of this fact walks through the pseudo-code and analyses every time information is added or 
deleted from each buffer, and it can be found in Section [TT] (see Lemma fl 1.21) . ■ 

Theorem 18.11 Except for the at most r? transmissions that may fail due to malicious activity, the 
routing protocol presented described in Sections [3 and enjoys linear throughput. More precisely, 
after x transmissions, the receiver has correctly outputted at least x — messages. If the number 
of transmissions x is quadratic in n or greater, than the failed transmissions due to adversarial 
behavior become asymptotically negligible. Since a transmission lasts 0{n^) rounds and messages 
contain 0{n^) bits, information is transferred through the network at a linear rate. 

Theorem 18.11 will follow immediately from the following three theorems. As with the proofs for 
the edge-scheduling protocol, line numbers for the pseudo-code have form (X.YY), where X refers 
to the Figure number, and YY refers to the line number. It will be convenient to introduce new 
terminology: 

Definition 10.1. We will say a node N ^ G participated in transmission T if there was at least one 
round in the transmission for which was not on the (sender's) blacklist. The sender's variable 
that keeps track of nodes participating in transmission T (|101 69) will be called the participating list 
for transmission T, denoted by V-y (it is updated at the end of every transmission on ll5l l87). 

Theorem 10.2. Every transmission (regardless of its success/failure) lasts O(n^) rounds. 

Proof. Line (|111 02) shows that each transmission, regardless of success or failure, lasts AD = O(n^) 
rounds. ■ 

Theorem 10.3. Suppose transmission T failed and at some later time (after transmission T but 
before any additional nodes have been eliminated) the sender has received all of the status report 
parcels from all nodes on Vi. Then the sender can eliminate a corrupt node. 

Proof. The proof of this theorem is rather involved, as it needs to address the three possible reasons 
(F2-F4) that a transmission can fail. It can be found in Section [lO.ll below. ■ 

Theorem 10.4. After a corrupt node has been eliminated (or at the outset of the protocol) and before 
the next corrupt node is eliminated, there can be at most n — 1 failed transmissions {Ti, . . . ,T„_i} 
before there is necessarily some index 1 < i < n — 1 such that the sender has the complete status 
report from every node on Vj- . 



72 



Proof. The theorem will follow from a simple observation: 

Observation. \i N ^Vj, then the sender is not missing any status report parcel for N for 
any transmission prior to transmission T. In other words, there is no transmission T' < T such 
that was blacklisted at the end of T' (as on I15L 188) and the sender is still missing status 
report information from N at the end of T. 

Proof. Nodes are added to the blacklist whenever they were participating in a transmission 
that failed (1151 187-88). Nodes are removed from the blacklist whenever the sender receives 
all of the status report information he requested of them (1141 164-166). or when he has just 
eliminated a node (|15L 171). in which case the sender no longer needs status reports from 
nodes for old failed transmission^ (and in particular, this case falls outside the hypotheses 
of the Theorem). Since V-y is defined as non-blacklisted nodes (|15L 187). the fact that N € Vj 
implies that was not on the sender's blacklist at the end of T. Also, notice the next line 
guarantees that all nodes not already on the sender's blacklist will be put on the blacklist 
if the transmission fails. Therefore, if A'' has not been blacklisted since the last node was 
eliminated (|15L 169-177). then there have not been any failed transmissions, and hence the 
sender is not missing any status reports. Otherwise, let T' < T denote the last time A^ was 
put on the blacklist, as on (|15L 188). In order for A'^ to be put on Vj on line (|15L 187) of 
transmission T, it must have been removed from the blacklist at some point between T' and 
the end of T. In this case, the remarks at the start of the proof of this observation indicate 
the sender is not missing any status reports from A^. ■ 

Suppose now for the sake of contradiction that we have reached the end of transmission T„_i, which 
marks the (n — 1)'^* transmission {Ti, . . . , T„_i} such that for each of these n — 1 failed transmissions, 
the sender does not have the complete status report from at least one of the nodes that participated 
in the transmission. Define the set S to be the set of nodes that were necessarily not on Pt„_i , and 
initialize this set to be empty. 

Since the sender is missing some node's complete status report that participated in Ti, there is 
some node A^i G Vj^ from which the sender is still missing a status report parcel corresponding to 
Ti by the end of transmission T„_i. Notice by the observation above that A^i will not be on Vj^ for 
any T2 < T' < T„_i, so put A''! into the set S. Now looking at T2, there must be some node N2 G Vj^ 
from which the sender is still missing a status report parcel from T2 by the end of transmission T^-i. 
Notice that A'2 7^ A'l since A^i ^ Vj^^ and also that A'2 ^ 'Pt„_i (both facts follow from the above 
observation), so put A'2 into S. Continue in this manner, until we have found the (n — 2)*^ distinct 
node that was put into S due to information the sender was still missing by the end of T„_2. But 
then \S\ = n — 2, which implies that all nodes, except for the sender and the receiver, are not on 
^T„_i (the sender and receiver participate in every transmission by Lemma lll.2ip . But now we 
have a contradiction, since Lemma fl 1 .221 says that transmission T„_i will not fail. ■ 

We are now ready to prove Theorem 18. 11 reserving the proof of Theorem ll0.3l to the next section. 

^'^The sender already received enough information to eliminate a node. Even though it is possible that other nodes 
acted maliciously and caused one of the failed transmissions, it is also possible that the node just eliminated caused 
all of the failed transmissions. Therefore, the protocol does not spend further resources attempting to detect another 
corrupt node, but rather starts anew with a reduced network (the eliminated node no longer legally participates), 
and will address future failed transmissions as they arise. 
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Proof of Theorem \8.1[ By Theorem 110.21 every transmission lasts at most O(n^) rounds, so it re- 
mains to show that there are at most failed transmissions. By Theorem 110.41 by the end of at 
most n — 1 failed transmissions, there will be at least one failed transmission T such that the sender 
will have all status report parcels from every node on Vj- Then by Theorem 110.31 the sender can 
eliminate a corrupt node. At this point, lines (I15L 169-177) essentially call for the protocol to start 
over, wiping clear all buffers except for the eliminated nodes buffer, which will now contain the 
identity of a newly eliminated node. The transmission of the latest codeword not yet transmitted 
then resumes (see comments on (I15L 214)). and the argument can be applied to the new network, 
consisting of n — 1 nodes. Since the node-controlling adversary can corrupt at most n — 2 nodes 
(the sender and receiver are incorruptible by the conforming assumption), this can happen at most 
n — 2 times, yielding the bound of for the maximum number of failed transmissions. ■ 

10.1 Main Technical Proof of the Node-ControlHng + Edge-Scheduling Protocol 

In this section, we aim to prove Lemma fl 0.31 which states that the sender will be able to eliminate 
a corrupt node if he has the complete status reports from every node that participated in some failed 
transmission T. We begin by formally defining the three reasons a transmission may fail, and prove 
that every failed transmission falls under one of these three cases. 

Theorem 10.5. At the end of any transmission T, ( at least) one of the following necessarily happens: 

SI. The receiver has received at least D — 6n^ distinct packets corresponding to the current codeword 

F2. SI does not happen, and the sender has knowingl\l^ inserted less than D packets 

F3. SI does not happen, the sender has knowingly^^ inserted at least D packets, and the receiver 

has not received any duplicated packets corresponding to the current codeword 
F4. SI, F2, and F3 all do not happen 

Proof. That the four cases cover all possibilities (and are disjoint) is immediate. Also, in the case 
of SI, the receiver can necessarily decode by Lemma [11.201 and hence that case corresponds to a 
successful transmission. Therefore, all failed transmissions must fall under one of the other three 
cases. ■ 

Note that case F2 roughly corresponds to packet duplication, since the sender is blocked from 
inserting packets in at least AD — D rounds, indicating jamming that cannot be accounted for by 
edge failures alone. Case F3 roughly corresponds to packet deletion, since the D packets the sender 
inserted do not reach the receiver (otherwise the receiver could have decoded as by Lemma ril.20|) . 
and case F4 corresponds to a mixed adversarial strategy of packet deletions and duplications. We 
treat each case separately in Theorems 110.61 [10.111 and 110.121 below, thus proving Theorem 110.31 

Proof of Theorem \1 0.31 Theorem 110.51 guarantees that each failed transmission falls under F2, F3, 
or F4, and the theorem is proven for each case below in Theorems 110.61 110.111 and 110.121 ■ 

We declare once-and-for-all that at any time, G will refer to nodes still a part of the network, 
i.e. nodes that have not been eliminated by the sender. 

^^Recall that by the definition of "inserted" (see l6.6l) . the sender may not have received confirmation (as in Definition 
17. 6p that a packet he outputted along some edge was received by the adjacent node. Case F3 requires that the sender 
has received confirmation for at least D packets. 
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Handling Failures as in F2: Packet Duplication 

The goal of this section will be to prove the following theorem. 

Theorem 10.6. Suppose transmission T failed and falls under case F2, and at some later time 
(after transmission T hut before any additional nodes have been eliminated) the sender has received 
all of the status report parcels from all nodes on Vj. Then the sender can eliminate a corrupt node. 

The idea of the proof is as follows. Case F2 of transmission failure roughly corresponds to 
packet duplication: there is a node N & G who is jamming the network by outputting duplicate 
packets. Notice that in terms of network potential (see Definition l6.10p . the fact that is outputting 
more packets than he is inputting means that will be responsible for illegal increases in network 
potential. Using the status reports for case F2, which include nodes' signatures on changes of 
network potential due to packet transfers and re-shuffling, we will catch N by looking for a node 
who caused a greater increase in potential than is possible if it had been acting honestly. The formal 
proof of this fact will require some work. We begin with the following definitions: 

Definition 10.7. The conforming assumption on the node-controlling and edge-scheduling adver- 
saries demand that for every round there is a path connecting the sender and receiver consisting of 
edges that are "up" and through uncorrupted nodes. We will refer to this path as the active honest 
path for round t and denote it by Pt, noting that the path may not be the same for all rounds. 

Definition 10.8. We will say that some round t (of transmission T) is wasted if there is an edge 
E{A, B) on that round's active honest path such that either Okay To Send Packet (I11L 31) or 
Okay To Receive Packet (I11L 35) returned false. 

Intuitively, a round is wasted if an edge on the active honest path was prevented from passing 
a packet either because one of the nodes was blacklisted or because there was important broadcast 
information that had to be communicated before packets could be transferred. 

Lemma 10.9. There are at most wasted rounds in any transmission T. 

Proof. We will prove this lemma via two claims. 

Claim 1. Every wasted round t falls under (at least) one of the following cases: 

1. An edge on Pt transfers 0t or a parcel of the sender's Start of Transmission (SOT) 
broadcast 

2. An edge on Pt transfers the label of a node to remove from the blacklist 

3. An edge on Pt transfers the information that one of the terminal nodes (on that edge) 
has the complete status report for a blacklisted node 

4. A node on Pt learns a status report parcel for a blacklisted node. More specifically, 
there is some node (A^, T',T) that was part of the SOT broadcast (i.e. the node began 
the transmission on the sender's blacklist) and some other honest node N G G such that 
A^ learns a new status report parcel from A^ corresponding to transmission T'. 

Proof. Let t be a wasted round. Denote the active honest path for round t by Pt = 
NqNi . . . Ni. By looking at Okay To Send Packet and Okay To Receive Packet (|111 31 
and I11L 35). we first argue that that cases 1-3 cover all possible reasons for a wasted round, 
except the possibility that one node is on the other's blacklist. To see this, we go through each 
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line of Okay To Send Packet and Okay To Receive Packet and consider what happens 
along a specified edge on Pt, noting that by assumption this edge is active and the neighboring 
nodes are honest, so the appropriate broadcast parcel will be successfully transferred (|111 15). 
In particular, it will be enough to show that for every reason a round may be wasted, there 
is a node on Pt that has broadcast information of type 1-4 (see line (|13L 115)) that it has yet 
to transfer across an adjacent edge on Pt, as then we will fall under cases 1-3 of the Claim. 

— If there is a node Ni on Pt that does not know all parcels of the S'OTbroadcast (|151 200). 
then find the last index < j < i such that Nj knows all of SOT but Nj+i does not (j 
is guaranteed to exist since S = Nq knows all of SOT and Ni does not). Then Nj has 
broadcast information of type 2 (I13L 115) it has not yet sent along its edge to Nj^i. 

— If there is a node Ni on Pt that knows ©t or all of SOT hut has not yet transferred one 
of these parcels across an edge of Pt, or Ni knows the complete status report for some 
blacklisted node N and Ni has not yet passed this fact along an edge on Pt, then Ni has 
broadcast information of type 1, 2, or 4 (|13L 115). 

— If there is a node Ni on Pt that knows of a node that should be removed from the 
blacklist, but it has yet to transfer this information across an edge of Pt, then Ni has 
broadcast information of type 3 (I13L 115). 

It remains to consider the final reason one of these two functions may return false, namely 
when there is some Ni on Pt that is on the blacklist of either A^j_i or A^j+i. Let BLs denote 
the sender's blacklist at the start of round t. 

— If Ni ^ BLs, then there will be some index < j < i + 1 such that at the start of round 
t, Ni is not on A'^'s blacklist but A', is on A'^+i's blacklist. We may assume that both 
Nj and Nj^i have received the full start of transmission broadcast, else we would be 
in one of the above covered cases. Since Ni is on Nj+is blacklist, Ni must have begun 
the transmission on the sender's blacklist (all internal nodes' blacklists are cleared at the 
end of each transmission (|151 203) and restored when they receive the SOT broadcast 
([I5l200), ([I4ll37-138)). However, since A^, is not on Nj's blacklist as of round t and Nj 
has received the full S'OTbroadcast, at some point in T, Nj must have received a parcel 
from the sender indicating Ni should be removed from the blacklist, as on (|141 147-149). 
Since A'^ and Nj^i are both honest and Nj has received the information that A'j should 
be removed from the blacklist (but Nj+i has noi received this information yet), it must be 
that this broadcast information of type 3 (|131 115) has not yet been successfully passed 
along E{Nj, Nj^i) yet. In particular, Nj has broadcast information of priority at least 
3 that he has yet to successfully send to Nj^i, so he will send a parcel of priority 1,2, 
or 3 in round t, which are in turn covered by Statements 1 and 2 of the Lemma. 

— If A^i € BLs, then there exist some < j < i such that Nj does not have A^j's complete 
status report, but A'^+i does (since N G BLs implies S does not have the complete 
status report, but Ni has its own complete status report in its broadcast buffer, see 
Statement 2 of Lemma fl 1.1 61) . Then if Nj+i has not yet passed the fact that it has such 
knowledge along E{Nj^i, Nj), then A'^+i had broadcast information of type 4, in which 
case we fall under case 3 of the Claim. On the other hand, if this information has already 
been passed along E{Nj-^-i, Nj), then Statement 4 of Lemma [11.161 implies that Nj is 
aware that Aj+i knows the complete status report of Ni (who by choice of j is on Nj^s 
blacklist), and hence a will necessarily be set as on (|13L 119 or ll3L 122) and sent to Nj on 
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(I13L 103)). Consequently, Nj+i will receive a (|131 105) during Stage 1 communication 
of round t, and will have broadcast information of type 5 (|131 115) it has not sent along 
E{Nj, Nj^i) yet. This broadcast parcel can then be sent in Stage 2 communication of 
round t (|111 15). and this is covered by case 4 of the Claim. ■ 

Claim 2. The maximum number of wasted rounds due to Case 1 of Claim 1 is n^, the 
maximum number of wasted rounds due to Case 2 of Claim 1 is n'^/2, the maximum number 
of wasted rounds due to Case 3 of Claim 1 is n^, and the maximum number of wasted rounds 
due to Case 4 of Claim 1 is n^. 
Proof. 

1. ©T is one parcel (|151 179). and the S'OTis at most 2n — 1 parcels (|151 200). so together 
they are at most 2n parcels. Since each honest node will only broadcast each of these 
parcels at most once across any edge (as long as the broadcast is successful, which it will 
be if the round is wasted due to Case 1) and there are at most n^/2 such edges, we have 
that Case 1 can happen at most times. 

2. Lemma 111.231 says that no honest node will accept more than one distinct parcel 
(per transmission) that indicates some node should be removed from the blacklist. 
Therefore, in terms of broadcasting this information, N will have at most one broadcast 
parcel per transmission per node N indicating N should be removed from the blacklist. 
Therefore, it can happen at most n times that an edge adjacent to an honest node will 
need to broadcast a parcel indicating a node to remove. Again since the number of edges 
is bounded by n^/2. Case 2 can be responsible for a wasted round at most n^/2 times. 

3. Lemma fl 1.241 says that for any node N € G that has received the full S'OT broadcast for 
transmission T, if A^ is honest then it will transmit along each edge at most once (per 
transmission) the fact that it knows some A^'s complete status report. Since each node 
has at most n — 1 adjacent edges and there are at most n nodes in G, Case 3 can be 
responsible for a wasted round at most times. 

4. Notice that Case 4 emphasizes the fact that a node on Pt learned a blacklisted node's 
status report parcel. Since there are at most n — 1 blacklisted nodes at any time (see 
(1151 187-188) and Claim [TL6]1 . and at most n status report parcels per blacklisted node 
(see (1141 142-45) and Lemma fll.Tp . an honest node can learn a new status report parcel 
at most n{n — 1) < times per transmission (see Statement 3 of Lemma 111.161 which 
says honest nodes will not ever "unlearn" relevant status report parcels). Since there are 
at most n nodes. Case 4 can be responsible for a wasted round at most times. ■ 

Claim 1 guarantees every wasted round falls under Case 1-4, and Claim 2 says these can happen at 
most 4n^ rounds, which proves the lemma. ■ 

We now define the notation we will use to describe the specific information the status reports 
contain in the case of F2 (see ^12), dSlT), 1^32), and ([l4H42-145))E!|: 

- SIGa,a denotes the net decrease in A's. potential due to re-shuffling packets in the current 
transmission. 

^^On a technical point, since our protocol calls for internal nodes to keep old codeword packets in their buffers from 
one transmission to the next, packets being transferred during some transmission may correspond to old codewords. 
We emphasize that the quantities in SIGa,a, SIG[2], and SIG[3] include old codeword packets, while S'/GIl] and 
SIG[p] do not count old codeword packets fsee llll ll and ll2l 59-60l. 
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- SIG'^'(1]a,b denotes the net increase in -B's potential due to packet transfers across directed 
edge E{A,B), as signed by B and stored in A's signature buffer ( (|121 75). (IllL ll). and 

(inio7)). 

- SIG'^\2\b,a denotes the net decrease in i?'s potential due to packet transfers across directed 
edge E{B,A), as signed by B and stored in A's signature buffer. Notice that SIG^[2]b^a is 
measured as a positive quantity, see lines (I12L 60). (I12L 62). and (I12L 74). 

- SIG^[3]a,b denotes the net decrease in A^s potential due to packet transfers across di- 
rected edge E{A,B), which is signed by A and stored its own signature buffer. Notice that 
SIG^[3]a,b is measured as a positive quantity, see line (|12L 49). 

- SIG^[3]b,a denotes the net increase in ^'s potential due to packet transfers across directed 
edge E{B,A), which is signed by A and stored its own signature buffer (I12L 75). 

Lemma 10.10. Suppose transmission T failed and falls under case F2, and at some later time (after 
transmission T hut before any additional nodes have been eliminated) the sender has received all of 
the status reports from every node on Vj. Then one of the following two things happens: 

1. There is some node A ^ G whose status report indicates that A is corrup¥^. 

2. There is some A ^ G whose potential at the start of T plus the net increase in potential during 
T is smaller than its net decrease in potential during T. More specifically, note that A's net 
increase in potential, as claimed by itself, is given by: 

SIG^[3]b,a 

BeV\A 

Also, A 's net decrease in potential, as documented by all of its neighbors and its own loss 
due to re-shuffling, is given by: 

SIGa,a+ E SW[2]a,b 

B&V\A 

Then case 2) says there exists some A G such that: 

4n3-4n2+ ^ SIG^[3]b,a < SIGa,a+ SIG''[2]a,b, (16) 

B£V\A BeV\A 

where the 4n^ — Av? term on the LHS is an upper bound for the maximum potential a node 
should have at the outset of a transmission (see proof of Claim [6^) . 

Proof. The idea of the proof is to use Lemma 111.121 which argues that in the absence of malicious 
activity, the network potential should drop by at least n every (non-wasted) round in which the 
sender is unable to insert a packet. Then since the sender could not insert a packet in at least 3D 

^"This includes, but is not limited to: 1) The node has returned a (value, signature) pair, where the value is 
not in an appropriate domain; 2) The node has returned non-zero values indicating interaction with blacklisted or 
eliminated nodes; 3)The node has reported values for SIG^[S\.s,a that are inconsistent with the sender's quantity 
SIG'^[2]s,a; or 4) The node has returned outdated information in their status report. By "outdated" information, we 
mean that as part of its status report, A returned a (value, signature) pair using a signature he received in round t 
from one of A's neighbors A'', but in A'''s status report. A*' provided a (value, signature) pair from A indicating they 
communicated after round t and that A was necessarily using an outdated signature from A^. 
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rounds (case F2 states the sender inserted fewer than D packets in the 4D rounds of the transmission) 
and since there are at most 4n^ wasted rounds per transmission, the network potential should have 
dropped by at least (n)(3-D — 4n^) > 2nD + 8n^ (since D = 6^ > I2n3 as A < 1/2). However, this 
is impossible, since the maximum network potential in the network at the start of the transmission 
(which is upper bounded from the capacity of the network) is 4n^ (Lemma I6.2p plus the maximum 
amount of network potential increase during transmission T is 2nD (since the sender inserted fewer 
than D packets at maximum height 2n), and hence the sum of these is less than 2nL' + 8n^, resulting 
in a negative network potential. Since network potential can never be negative, there must be illegal 
increases to network potential not accounted for above, and the node responsible for these increases 
is necessarily corrupt. We now formalize this argument, showing how to find such an offending node 
and prove it is corrupt. 

Let (3 denote the number of rounds in transmission T that the sender was blocked from inserting 
any packets, and V denote the participating list for T. 

Obs. 1: If there exists A ^ V such that one of the following inequalities is not true, then A is 



Proof. The above inequalities state that for honest nodes, the potential changes due to re- 
shuffling and packet transfers are strictly non-positive (this was the content of Lemma fG. 11(1 . 
This observation is proved rigorously as Statements 4 and 5 of Lemma 111.91 in Section [TT] □ 

Obs. 2: The increase in network potential due to packet insertions is at most 2nD + 211? . More 
precisely, either there exists a node A ^ G such that the sender can elinninate A, or the following 
inequality is true: 



Proof. By hypothesis, the sender knowingly inserted less than D packets in transmission T, 
and each packet can increase network potential by at most 2n. The sum on the LHS of 
(fTTl) represents the increase in potential claimed by nodes participating in T caused by packet 
insertions. This quantity should match the sender's perspective of the potential increase 
(which is at most 2nD), with the exception of potential increases caused by packets that 
were inserted but S did not received confirmation of receipt (see Definition I7.6|l . There can 
be at most one such packet per edge, causing an additional potential increase of at most 2n 
per edge. Adding this additional potential increase to the maximum increase of 2nD of the 
sender's perspective is the RHS of (fT7|l . The formal proof can be found in Lemma [11.151 in 
Section [fll □ 

Obs. 3: /? > 3D — n. (Recall that (5 denotes the number of blocked rounds in T.) 

Proof. Since the sender knowingly inserted fewer than D packets, there could be at most n 
packets (one packet per edge) that was inserted unbeknownst to 5, and hence the sender must 
have been blocked for (at least) all but D -|- n of the rounds of the transmission. Since the 
number of rounds in a transmission is AD (I11L 02). we have that [5 > 3D — n. □ 



corrupt. 



< SIGa,a < ^ {SIG^[2]b,a-SIG^[3]b,a) 



B£V\{A,S} 




(17) 



Aev\s 
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Let Ti-Y C Vt denote the subset of participating nodes that are honest (the sender is of course 
obhvious as to which nodes are honest, but we will nevertheless make use of TLj in the following 
argument). For notational convenience, since transmission T is fixed, we suppress the subscript and 
write simply TL and V. We make the following simple observations: 

Obs. 4: The following inequality is true: 

2nD + - 4n3 + 2n2 < ^SIGa,a + Y^ ^ {SIG^[2]b,a - SIG^[?,]b,a) (18) 

A&H\S AeH\S B&V\{A,S} 

Proof. This follows immediately from Observation 3 and Lemma [11.121 since: 

n{(3i - W) > n{2,D -n- 4n^) 

> 2nD + An^ - 4n^ + 2n^ 

where the first inequality is Observation 3, and the second follows because D = 6nVA > 
8n^ > 8n^ - 4n^ + 3n. □ 



Obs. 5: Either a corrupt node can be identified as in Obs. 1 or 2, or there is some A such 
that: 

Arv" -An" <SIGa,a+ Y1 SIG''[2]a,b - SIG^[3]b,a (19) 

B£V\A 

Proof. Consider the following inequalities: 

2/11) + 4n^ - 4n3 + 2n2 < Y,SIGa,a+Y1 Y1 iSIG'^[2]B,A - SIG'^[3]b,a) 

AeH\S AeH\S BeV\{A,S} 

< J2sIGa,a+Y. E iSIG^[2]B,A-SIG^[3]BA) 

AeV\S AeP\S B&V\{A,S} 

= ^SIGa,a+Y1 iSIG''[2]A,B-SIG^[3]B,A) (20) 

Aev\s AeP\s Bev\{A,s} 

Above, the top inequality follows from Obs. 4, the second inequality follows from Obs. 1, and 
the third line is a re-arranging and re-labelling of terms. Subtracting 2nD + 2r? from both 
sides: 

4n^-4n3< ^ 5/Ga,a + {S1G^\2\a,b - S1G\3\b,a) - 2nD - 2n^ 

Aev\s Aep\s Bev\{A,s} 

< Y.SIGa,a+Y1 E iSIG''[2]A,B-SIG^[3]B,A)+ -5/G^[3]5,a 
Aev\s AeP\s Bev\{A,s} A&v\s 

= J2sIGa,a+Y1 E iSIG''[2]A,B-SIG^[3]B,A) + 

AGr\S AeP\S BeP\{A,S} 

iSIG^[2]A,s-SIG^[3]s,A) 

Aev\s 

= Y.SIGa,a+Y1 E ('^^^''[2]a,b-5/G^[3]b,a) (21) 

AeV\S A&V\S BeV\A 
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Above, the top inequality is from ([20]) . the second follows from Obs. 2, the third line is because 
SIG^[2]a,s = for all A G G (5 never receives a packet from anyone, see (I11L 21-22)). and 
the final line comes from combining sums. Using an averaging argument, this implies there is 
some A € V \ S such that: 

4n^-4n^ <SIGa,a+ {SIG''[2]a,b - SIG^[3]b,a), (22) 

B£P\A 

which is ([19]). □ 

Therefore, if a node cannot be eliminated as in Obs. 1 or 2 (which are covered by Case 1 of Lemma 
110.101) . then Obs. 5 implies that Case 2 of Lemma flO-lOl is true. ■ 

Proof of Theorem \10.6l This Theorem now follows immediately from Lemma 110.101 and the fact 
that a node A G G for which (fTHIl is true is necessarily corrupt. Intuitively, such a node A € G 
is corrupt since the potential decrease at A is higher than can be accounted for by A's potential 
at the outset of T plus the potential increase due to packet insertions from the sender. The formal 
statement and proof of this fact is the content of Corollary 111.141 ■ 

Handling Failures as in F3: Packet Deletion 

The goal of this section will be to prove the following theorem. 

Theorem 10.11. Suppose transmission T failed and falls under case F3, and at some later time 
(after transmission T hut before any additional nodes have been eliminated) the sender has received 
all of the status report parcels from all nodes on Vj. Then the sender can eliminate a corrupt node. 

The idea of the proof is as follows. Case F3 of transmission failure roughly corresponds to packet 
deletion: there is a node N & G who is deleting some packets transferred to it instead of forwarding 
them on. Using the status reports for case F3, which include nodes' signatures on the net number 
of packets that have passed across each of their edges, we will catch N by looking for a node who 
input more packets than it output, and this difference is greater than the buffer capacity of the 
node. 

Proof. We first define the notation we will use to describe the specific information the status reports 
contain in the case of F3 (([9117), ^32), and ([I4ll44)): 

- SIG^[1]a,b denotes the net number of packets that have travelled across directed edge 
E{A,B), as signed by B and stored in A^s (outgoing) signature buffer. 

- SIG^[1]b,a denotes the net number of packets that have travelled across directed edge 
E{B,A), as signed by B and stored in j4's (incoming) signature buffer. 

By the third Statement of Lemma fl 1.131 either a corrupt node can be eliminated, or the following 
is true for all A,B & G: 

\SIG^[1]b,a - SIG^[1]b,a\ < 1 and \SIG^[1]a,b - SIG^[1]a,b\ < 1 
Then summing over all A,B G V: 

\SIG^[1]b,a-SIG''[1]b,a\ < (23) 

A,Be.V,A¥'B 
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This in turn imphes that: 

-n" < Yl {SIG''[1]b,a-SW[1]b,a) 

A,B(^V,A¥'B 
A£V B£V\A 

= J2 sig''[i]b,r - Yl siG''[i]s,B + 

BeV\R BeV\S A£V\{R,S}BeP\A 

< _6n3 + Y Y^^^^^^^^B'^ - SIG^[1]a,b) 

AeP\{R,S}B&V\A 

The first inequahty is from ([23I1 . the second hne is from re-labelhng and re-arranging terms, the 
third hne comes from separating out the terms A = S and A = R and noting that SIG^[1]r^b = 
SIG^[1]b,s = (since the receiver will never output packets to other nodes and the sender will 
never input packets, see (|lll l6-20) and (1111 21-22)). and the final inequality is due to the fact 
that we are in case F3, so the sender knowingly inserted D packets, but the receiver received fewer 
than D — 6n^ packets corresponding to the current codeword^. Using an averaging argument, we 
can find some A £ G such that: 

4n2-8n<6n2-n< Y iSIG'^[l]B,A - SIG'^[1]a,b), (24) 

BeV\A 

where the first inequality is obvious. Statement 7 of Lemma 1 1 1 . 91 now guarantees that A is corrupQ. 



Handling Failures as in F4: Packet Duplication + Deletion 

The goal of this section will be to prove the following theorem. 

Theorem 10.12. Suppose transmission T failed and falls under case F4, and at some later time 
(after transmission T but before any additional nodes have been eliminated) the sender has received 
all of the status report parcels from all nodes on Vj. Then the sender can eliminate a corrupt node. 

The idea of the proof is as follows. Case F4 of transmission failure roughly corresponds to packet 
duplication and packet deletion: there is a node N £ G who is replacing valid packets with copies of 
old packets it has already passed on. Therefore, simply tracking potential changes and net packets 
into and out of will not help us to locate N, as both of these quantities will be consistent with 
honest behavior. Instead, we use the fact that case F4 implies that the receiver will have received 
some packet p (from the current codeword) twice. We will then use the status reports for case F4, 
which include nodes' signatures on the net number of times p has crossed each of their edges, to 
find a corrupt node N by looking for a node who output p more times than it input p. 

^^More precisely, F3 states that the sender knowingly inserted at least D packets and the receiver did not receive 
any packet (from the current codeword) more than once. By Fact 1', since we are not in case SI, the receiver got 
fewer than D — 6n"^ distinct packets corresponding to the current codeword. 

^^Intuitively, A must be corrupt since the sum on the RHS of l[24|l represents the net number of packets A input 
minus the number of packets A output. Since this difference is larger than the capacity of ^'s internal buffers, A 
must have deleted at least one packet and is necessarily corrupt. 
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Proof. By definition of F4, the receiver received some packet p (corresponding to the current code- 
word) at least twice. Therefore, when (1151 178-179) is reached, the receiver will send the label of 
p back to the sender (which reaches S by the end of the transmission by Lemma I11.19P , and this is 
in turn broadcasted as part of the sender's start of transmission broadcast in the following trans- 
mission ( (I15L 190-192) and (|15L 200)). We will use the following notation to describe the specific 
information the status reports contain in the case of F4 (see ([9117) and ([9l32)): 

- SIG^\p]a,b denotes the net number of times p has travelled across directed edge E{A,B), as 
signed by B and stored in ^'s (outgoing) signature buffer. 

- SIG^\p]b,a denotes the net number of times p has travelled across directed edge E{B,A), as 
signed by B and stored in A^s (incoming) signature buffer. 

Consider the following string of equalities: 

= Yl E(^^^^Mb,a-5/G'^Mb,a) 

= E E('^^'^''Wab-'5/G^Nb,a) 

Y.^SIG''[p]a,b - SIG\]b,a) + iSIG''[p]B.,B - SIG'^[p]b,r) + 
AeVT\{R,s} BeVT BeVj 

{SIG^[p]s,B-SIG''[p]B,s) 

BeVT 

E Y.^SIG''[p]a,b-SIG%]b,a)+ Y^^^^''^P^s,b-SIG''\p]b,r) (25) 
AeVT\{R,S} BePi BeVj 

The first equality is trivial, the second equality comes from re-labelling and rearranging the terms 
of the sum, the third comes from separating out the A = S and A = R terms, and the final equality 
results from the fact that R never outputs packets and S never inputs packets, and hence they will 
never sign non-zero values for SIG\p]r^b or SIG\p]b,s, respectively (see (I11L 16-20) and (|11L 21- 
22)). Because p was received by R at least twice (by choice of p) and S will never send any packet 
to more than one nodS, we have that: 

Y iSIG''\p]s,B - SIG''\p]b,r) < -1 (26) 
BeVj 

Plugging this into ([25]l and rearranging: 

1^ E J2(^IG''\p]a,b-SIG%]b,a) (27) 

A€Vt:\{R,S} BeVi 

By an averaging argument, there must be some A ^Vt\ {R, S} such that: 

1 < {SIG''\p\A,B - SIG%]b,a) (28) 
BeVj 

Now Statement 8 of Lemma fl 1.91 says that A is necessarily corrupiEll. ■ 

^^This was proven in Observations 2-3 of Lemma 17.131 for the edge-scheduling protocol. However, the proofs of 
these observations remain valid in the (node-controlling-|-edge-scheduling) model because the sender is honest (by 
the conforming adversary assumption). 

^^Intuitively, A is corrupt since II28II says that it has output p more times than it input p. 
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11 Node-Controlling + Edge- Scheduling Protocol: 
Pseudo-Code Intensive Proofs 



In this section, we give detailed proofs that walk through the pseudo-code of Figures M - US] 
to argue very basic properties the protocol satisfies. The following lemma will relieve the need to 
re-prove many of the lemmas of Sections [6] and El 

Lemma 11.1. Differences between the edge-scheduling adversary protocol and the (node- controlling 
+ edge- scheduling) adversary protocol all fall under one of the following cases: 

1. Extra variables in the Setup Phase 

2. Length of transmission and codeword being transmitted for the current transmission 

3. Need to authenticate signatures on packets, as on 1111 08 ) and 1121 68 ) 

4- Need to check if it is okay to send/receive packets, as on 1121 59 ) and 1121 63 ) 

5. Broadcasting information, i.e. transmission of broadcast parcels and modifications of Broadcast 
Buffer, Data Buffer, and Signature Buffer 

Furthermore, differences as in Cases 3 and 4 O'l'^ identical to having an edge-fail in the edge- 
scheduling adversary protocol. Also, differences as in Case 5 affect the routing protocol only insofar 
as their affect on Cases 3 and 4 above. Furthermore, between any two honest nodes, the authen- 
tications of Case 3 never fail, and Case 4 failures correspond to "wasted" rounds (see Definition 

HEM)- 

Proof. Comparing the pseudo-code of Figures [5j [6l and [7] to Figures [1X11121 and 1151 as emphasized 
by line numbers in bold face in the latter three, it is clear that all differences fall under Cases 1-5 
of the lemma. Also, all of the other methods in Figures [T31I15I fall under Cases 4 and 5. 

As for the differences as in Cases 3 and 4, it is clear that failing Verify Signature One on 
(|12l 86-87) is equivalent to the edge failing during Stage 2 (i.e. as if p = _L on (I12L 62) causing 
(|121 69) to fail); failing Verify Signature Two on (1121 89-90) is equivalent to the edge failing 
during Stage 1 (since this sets HfN and RR to _L on (I12L 90). which is equivalent to the commu- 
nication on jSlOe) not being received); failing Okay to Send Packet on (I12L 59) is equivalent to 
the edge failing during Stage 2 (so that nothing is received on lines (lllL 22/ fT2l 62)): and failing 
Okay to Receive Packet on (I12L 63) is equivalent to the edge failing during Stage 1 (i.e. as if 
nothing is received on (I11L 13). so that Hqut = -L on (I12L 63)). Finally, differences as in Case 5 
do not directly affect routing (except their affects captured by Cases 3 and 4) since the transfer of 
broadcast parcels and maintenance of the related buffers (signature, broadcast, and data buffers) 
happen independently of the routing of codeword packets. This is evident by investigating the 
relevant bold lines in Figures llll 1121 and 1151 

The second part of the last sentence is true by definition of wasted (see Definition 110. Sp . and 
the first part follows from lines (HHH), (Il2l49), (Il2l60), (US 75), and Lemma [UlTl ■ 

Lemma 11.2. The domains of all of the variables in Figures\^ and (221 are appropriate. In other 
words, the protocol never calls for more information to be stored in an honest node 's variable (buffer, 
packet, etc.) than the variable has room for. 
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Proof. The proof for variables and buffers that also appear in the edge-scheduling protocol follows 
from Lemmas 17.11 and [7.21 since all differences between the edge-scheduling protocol and the (node- 
controlling -|- edge-scheduling) protocol are equivalent to an edge- failure (Lemma lll.lj) . So it 
remains to prove the lemma for the new variables appearing in Figures [9] and [10] (i.e. the bold line 
numbers). The distribution of public and private keys ([91.14) is performed by a trusted third party, 
so these variables are as specified. Below, when we refer to a specific node's variable, we implicitly 
assume the node is honest, as the lemma is only concerned about honest nodes. 

Bandwidth P ([9l06). We look at all transfers along each directed edge in each stage of any 
round. In Stage 1, this includes the transfer of Hqut, Hpp, FR (|111 06). Cbp, a (|111 04). 
and the seven signed items on (IllL ll). All of these have collective size 0{k + logn) (([9l03- 
04), ([9121), ([9123), ([9124), ^27), ^29), ^32), ([9l35), and ([9l36)). In Stage 2, this 
includes the transfer of the seven items on (|12L 60) and bp (|11L 15). Collectively, these have 
size 0{k + logn) (([9103-04), ^17), ^18), ^21), and ^42)). 

Potential Lost Due to Re-Shuffling SIGn,n ([9ll2). This is initialized to zero on (|101 46). after 
which it is only updated on ^76), ([ISSO), ([12180), ([12182), ([141128), ([141133), ([141141), 

and (I14L 146). The first four of these increment SIGn,n by at most 2re, and the latter four all 
reset SIGn,n to zero. We will see in Lemma fl 1 . 161 below that SIGn^n will always represent 
the potential lost due to re-shuffling in at most one failed transmission, and consequently 
SIGn^n is polynomial in n, as required. 

Broadcast Parcel bp to Receive ([91.27). This is initialized to -L on (I10L 58). after which it is 
only updated on (I13L 93). Either no value was received on (I13L 93) (in which case 6p = _L), 
or it corresponds to the value sent on (|131 97). As discussed below, the value of bp sent on 
(I13L 97) lies in the appropriate domain, and hence so does bp. 

Broadcast Buffer Request a ([9l28). This is initialized to _L on (I10L 58). after which it is only 
updated as in Broadcast Parcel to Request (|13L 117-122). On (I13L 117). a is set to _L, 
and on (|131 119) and (|131 122). a includes the label of a node and a status report parcel (see 
1141 142-145). and so a is bounded by 0{k + logn) = P as required. 

Outgoing Verification of Broadcast Parcel Bit Cbp ([9l29). This is initialized to zero on (|101 58). 
after which it is only updated as on (I12L 40) and (|13l lll). where it clearly lies in the appro- 
priate domain. 

Broadcast Parcel bp to Send ([9l42). This is initialized to _L on (|101 52). after which it is 
only updated as in Determine Broadcast Parcel to Send (I13L 115). Looking at the six 
types of broadcast parcels on line (I13L 115) and comparing the corresponding domains of these 
variables in Figures [9] and llOl we see that in each case, bp can be expressed in 0{k+\ogn) = P 
bits. 

Incoming Verification of Broadcast Parcel Bit Cbp ([9l43). This is initialized to zero on (|101 52). 
after which it is only updated as on ([ISlOS) and ([ISlOS). The value it takes on ([131105) 
will either be set to zero (if no value was received), or it will equal the value of Cbp sent on 
(I13L 101). which as shown above is either a one or zero. Meanwhile, the value it takes on 
(I13L 108) is zero, so at all times Cbp equals one or zero, as required. 
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First Parcel of Start of Transmission Broadcast (|10L 66). This is initialized to (0,0,0,0) on 
^72) and is only changed on ([151174), ^184), ^192), ([151195), and ([111198). In 

all of these cases, it is clear that can be expressed in O(logn) bits, as required. 

Number of Rounds Blocked l3j (I10L 67). This is initialized to zero on (I10L 71) and is only 
changed on STT\27), (fT5ll72). and ([13201). Notice that in the latter two cases, /3t is 
reset to zero, while can only be incremented by one on (|111 27) at most iD times per 
transmission by (I11L 02). Since either line (|151 172) or line (|151 201) is reached at the end of 
every transmission (in the case a node is not eliminated as on line (|14L 163) or (I14L 168). line 
([I5l201) will be reached by the call on ([lll29)), /Sj € [O.AD] at all times, as required. 

Number of Failed Transmissions F (|101 68). This is initialized to zero on (I10L 71) and is only 
changed on (I15L 172) and (|151 186). Notice that F is only incremented by one as on line 
(I15L 186) when a transmission fails. As was shown in Theorem 110.41 there can be at most 
n — 1 failed transmissions before a node can necessarily be eliminated, in which case F is reset 
to zero on ([151172). 

Participating List Vj (I10L 69). This is initialized to G on (|101 73) and is only changed on 
(I15L 173) and (|151 187). where it is clear each time that Vj G in both places. 

End of Transmission Parcel Gt (I10L 77). This is initialized to _L on (|101 79) and is only changed 
on (|151 179). where it is clear that Ox can be expressed in 0(A;+logn) bits as required (packets 
have size 0{k + logn), and the index of a transmission requires O(logn) bits). 

Broadcast Buffer BB ([9l08). We treat the sender's broadcast buffer separately below, and 
consider now only the broadcast buffer of any internal node or the receiver. Notice that the 
broadcast buffer is initially empty (I10L 46). Looking at all places information is added to 
BB (lines ([131106-107), ([11125), ([111127), ([MlSO), ([11136), ([11138), ([11142-145), 
([11148-149), (fT4ll54l. and ([11155)), we see that there are 7 kinds of parcels stored in the 
broadcast buffer, as listed on (|131 115) (the 7^^ type is to indicate which parcel to send across 
each edge, as on (|13L 106)). We look at each one separately, stating the maximum number of 
bits it requires in any broadcast buffer. For all of the items below, the comments on (I14L 123) 
ensure that there are never duplicates of the same parcel in BB at the same time, and also 
that every parcel in BB has associated with it n — 1 bits to indicate which edges the parcel 
has travelled across (see e.g. ([131107), ([11125), ([11127), ([11130), ([11136), ([11138), 
(I14L 148). and (I14L 154)). Totaling all numbers below, we see that the BB needs to hold at 
most n? + 5n broadcast parcels, with each parcel needing to record which of the n — 1 edges 
it has traversed, which proves the domain on ([9l08) is correct. 

1. Receiver's End of Transmission Parcel Gj. This is added to a node's broadcast 
buffer on (|141 125). and removed on (I15L 203). Since every internal node and the receiver 
will reach (|151 203) at the end of every transmission ( (I11L 30) and (|15L 203)). and by 
the inforgibility of the signature scheme, there is only one valid Gj per transmission T. 
Therefore, each node will have at most one broadcast parcel of this type in BB at any 
time. 

2. Sender's Start of Transmission Parcels. These are added to a node's broad- 
cast buffer on ([lll27), ([11130), ([11136), and ([11138), and they are removed on 
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(I15L 203). Since every internal node and the receiver will reach (I15L 203) at the end of 
every transmission ( (|111 30) and (I15L 203)). by the inforgibility of the signature scheme, 
for every transmission T, there is only one valid in BB at any time. Notice that ^Ij 
can hold up to 1 parcel for 200a, n — 1 valid parcels for 200b and 200d together, and up 
to n — 1 valid parcels for 200c (see (|151 200). and use the fact that S ^ EN,BL and 
Theorem I10.4p . Therefore, each node will have at most 2n broadcast parcels of this type 
in BB at any time. 

3. Label of a Node to Remove from the Blacklist. Parcels of this nature are added 
to a node's broadcast buffer on (I14L 148) and removed on (I15L 203). Since every internal 
node and the receiver will reach (I15L 203) at the end of every transmission ( (I11L 30) 
and (I15L 203)). we argue that in any transmission, every node will have at most n — 1 
parcels in their broadcast buffer corresponding to the label of a node to remove from 
the blacklist. To see this, we argue that the sender will add (iV,0,T) to his broadcast 
buffer as on (I14L 165) at most once for each node N £ V\S per transmission, and then 
use the inforgibility of the signature scheme to argue each node will add a corresponding 
broadcast parcel to their broadcast buffer as on (|14L 148) at most n — 1 times. That the 
sender will enter line (I14L 165) at most once per node per transmission is clear since once 
the sender has reached (I14L 165) for some node A^, the node will be removed from his 
blacklist on (I14L 166). and nodes are not re-added to the blacklist until the end of any 
transmission, as on (I15L 188). Therefore, once the sender has received some node A^'s 
complete status report as on (I14L 164). that same line cannot be entered again by the 
same node A'^ in the same transmission. In summary, there are at most n — 1 broadcast 
parcels of this type in any node's broadcast buffer at any time. 

4. The Label of a Node N Whose Status Report is Known to A^. We show that 
for any node N £V\S, there are at most (n — l) broadcast parcels of type 4 (|131 115) in 
BB at any timj^. This follows from the same argument as above, where it was shown 
that (|141 164) can be true at most once per node per transmission. The inforgibility 
of the signature scheme ensures that the same will be true for internal nodes regarding 
line (|14L 155). and since this is the only line on which broadcast parcels of this kind are 
added to BB, this can happen at most n — 1 times per transmission. However, we are 
not yet done with this case, because broadcast information of this type is not removed 
from BB at the end of each transmission like the above forms of broadcast information. 
Therefore, we fix A^ G G, and show that if A'' adds a broadcast parcel to BB of form 
(A^, A^, T') as on (I14L 155) of transmission T, then necessarily BB was not already storing 
a broadcast parcel of form {N,N,'l") for some other T" ^ T' (if T" = T', then there is 
nothing to show, as nothing new will be added to BB by the comments on I14L 123). 

For the sake of contradiction, suppose that BB is already storing a parcel of form 
{N,N,T") when (I14L 155) of transmission T is entered and A^ is called to add (A^, A^,T') 
to BB for some T' 7^ T". Since (|14L 155) is reached, we must have that (|14L 152) was 
satisfied for the bp appearing there. In particular, A^ is on A^'s version of the blacklist. 
Since the blacklist is cleared at the end of every transmission (|151 203). it must be 
that (A^, T',T) was added to A^'s version of the blacklist during the SOT broadcast for 
the current transmission T, as on (|14L 137-138). Therefore, all parcels in BB of form 

The (n — 1) comes from the fact that there are no status reports for the sender. 
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{N, N, T") for T" 7^ T' should have been removed from BB on Hne (|141 139). yielding the 
desired contradiction. 

5-6. Status Report Parcels. We fix iV g G and show that for every N e V \ {S,N}, 
there are at most n status report parcels corresponding to N in A'^'s broadcast buffer, 
and hence A^'s broadcast buffer will hold at most n(n — 1) status report parcels at any 
time. Since a single node's status report for a single transmission consists of at most n 
parcels (see lines (|14L 142-145i^). it will be enough to show that for every N G V \ S , 
at all times A^'s broadcast buffer only holds status report parcels for N corresponding to 
a single failed transmission T'. 

For the sake of contradiction, suppose that during some transmission T, there is some 
node N & V \ S and two transmissions T' and T" such that A^'s broadcast buffer holds 
at least one status report parcel for A^ from both T' and T". Notice that status report 
parcels are only added to BB as on (I14L 154). and without loss of generality suppose 
that the status report parcel of A^ corresponding to T" was already in BB when one 
corresponding to T' is added to BB as on (|141 154) of transmission T. As was argued 
above, since (|14L 154) is reached in T, (I14L 152) must have been satisfied, and since A^'s 
blacklist is cleared at the end of every transmission (|15L 203). it must be that a broadcast 
parcel of form (A^, T, T), adding A'^ to A'"'s version of the blacklist, was received earlier 
in transmission T. Notice that necessarily T = T', since otherwise line (I14L 153) will not 
be satisfied. But then since T" / T', all status report parcels of A^ corresponding to 
transmission T" should have been removed from BB on (I14L 139). yielding the desired 
contradiction. 

Now for a A^'s own status report parcels, these are added to BB on (|141 142-145). 
Investigating lines (fT4l l37l. p^ l39). and (fT4l l40l. we see that status reports of A^ 
can occupy BB for at most one failed transmission. 

7. Requested Parcel for Each Edge. For any edge E{A,N), N will have at most 
one copy of a parcel like a as on (|131 106) at any time, since the old version of a is 
simultaneously deleted when the new one is added on (I13L 106). Since each node has 
(n — 1) incoming edges, BB need hold at most n — 1 parcels of this form at any time. 

Data Buffer DB, Eliminated List ^A^, and Blacklist BL ([9l09-ll). We treat the sender's 
broadcast buffer separately below, and consider now only the data buffer of any internal node 
or the receiver. The data buffer (which includes the blacklist and list of eliminated nodes) is 
initially empty (I10L 46). A node A^'s data buffer holds three different kinds of information: 
blacklist, list of eliminated nodes, and for each neighbor B £ G, a list of nodes A^ G G for 
which B knows the complete status report (see item 4 on line (|131 115)). Below, we show 
that these contribute at most n — 1, n — 1, and (n — 1)^ parcels (respectively), so that DB 
requires at most parcels at any time. 

Blacklist BL. Each entry of BL is initialized to _L on (fT0l 46). and BL is only modified 
on lines ([I4ll34), ([I4ll38), ([I4ll48), and ([151203). BL is an array with n-1 entries, 
indexed by the nodes mV\S. When a node (A^, T) is added to BL as on (|141 138). this 
means that the entry of BL corresponding to A^ is switched to be T. When a node (A^, T) 

^®We assume that the signature buffer information for two directed edges E{A,B) and E{B,A) are combined into 
one status report parcel. 
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is removed from BL as on (|141 148). this means that the entry of BL corresponding to 
is switched to _L. Finally, when BL is to be cleared as on (I14L 134) and (I15L 203). this 
means that BL each entry of BL is set to _L. Thus, in all cases, BL E [l..n — 1] x {0, l}^ 
as required. 

List of Eliminated Nodes EN. Each entry of EN is initialized to -L on p!0l 46l. 
and is only modified on line (I14L 132). EN is an array with n — 1 entries, indexed by 
the nodes in 7^ \ 5. Here, when a node {N, T) is added to EN, this means the entry of 
EN corresponding to is switched to T. Thus, at all times EN € [l..n — 1] x {0, l}^ as 
required. 

Which Neighbor's Know Another Node's Status Report. Parcels of this kind 
are only added to or removed from DB on lines p^ l34). (fT4ll39). (fT4ll49l. and 
(I14L 151). We will now show that for any pair of nodes N,B & V \ S, the data buffer 
of any node N £ G will have at most one parcel of the form {B,N,T'), from which we 
conclude that this portion of A^'s data buffer need hold at most (n — 1)^ parcels. To 
see this, we fix B and A^ in G and suppose for the sake of contradiction that A^'s data 
buffer holds two different parcels {B,N,T') and (5,iV,T"), for T' / T". We consider the 
transmission T for which this first happens, i.e. without loss of generality, (-B,A^, T') is 
added to DB as on (|14L 151) of T. Since the second part of (I14L 151) is reached, the 
first part of (|141 151) must have been satisfied, and since the blacklist is cleared at the 
end of every transmission (|151 203). it must be that a broadcast parcel of form (A/^, T, T) 
adding A^ to A^'s version of the blacklist was received earlier in transmission T. Notice 
that necessarily T = T', since otherwise line (|141 151) will not be satisfied. But then since 
T" 7^ T', {B,N,T") should have been removed from DB as on (|141 139) of transmission 
T, yielding the desired contradiction. 

Adding these three contributions, we see that that DB requires at most parcels, as required. 

Outgoing Signature Buffers SIG JSlT). Each outgoing signature buffer is initially empty 
([I0l54), and they are only modified on ([l2l48-49), 1^128), ((11133), ((11141), and 

(I14L 146). The first of these increments 5/G[3] by at most 2n, increments SIG[1], and SIG[p] 
by at most 1, and increments SIG[2] by at most 2n, and the latter four lines all reset all entries 
of SIG to _L. Since our protocol is only intended to run polynomially-long (in n), each entry 
of SIG is polynomial in n, as required. 

Incoming Signature Buffers SIG ((9l32). Each incoming signature buffer is initially empty 
^48), and they are only modified on ((12174-75), ((11128), ((T1133), ((T1141), and 

((11146). The first of these increments SIG[3] by at most 2n, SIG[1] and SIG\p] by at 
most 1, and SIG[2] by at most 2n, and the latter four lines all reset all entries of SIG to 
_L. Since our protocol is only intended to run polynomially-long (in n), each entry of SIG is 
polynomial in n, as required. 

Copy of Packets Buffer COPY 1^60). GOPY is first filled on ((I0l74)/((l5l214), with a 
copy of every packet corresponding to the first codeword. The only place it is modified after 
this is on (|151 214). where the old copies are first deleted and then replaced with new ones. 
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Sender's Broadcast Buffer BB. In contrast to an internal node's broadcast buffer, the only thing 
the sender's broadcast buffer holds is the Start of Transmission broadcast (I15L 200) and the 
information that a node should be removedfrom the blacklist, see (I14L 165). Notice that at the 
outset of the protocol, BB only holds the Start of Transmission broadcast, which is comprised 
by only Qi = (0, 0, 0, 0) (1101 72-73). After this, the only changes made to BB appear on lines 
dlllieS), ([I5ll71), ((I5ll99), and ([I5l200). Notice that for every transmission, necessarily 
either (|15L 171) or (I15L 199) will be reached, and hence at any time of any transmission T, BB 
contains parcels corresponding to at most one Start of Transmission broadcast, and whatever 
parcels were added to BB so far in T. By investigating line (I15L 200) and using Lemma ri0.4l 
the former requires at most 2n parcels, and by the comment on (|141 156). the latter requires 
at most n parcels (|14L 165). Therefore, the sender's broadcast buffer requires at most 3n 
parcels, as required. 

Sender's Data Buffer DB, Eliminated List EN, and Blacklist BL ([10162-64). We will show that 
the sender's DB needs to hold at most + + n parcels at any time, and that the blacklist 
and list of eliminated nodes need at most n parcels each. Notice that every entry of DB is 
initialized to _L on (I10L 73). after which modifications to DB occur only on lines (|141 158). 
([141160), ([11162), ([141166), ([111170), ^171), ([111187), ([IHlSS), ([111191), ([111194), 
(I15L 197). and (I15L 199). The sender's data buffer holds eight different kinds of information: 
end of transmission parcel ©x, status report parcels, the participating list for up to n — 1 
failed transmissions, the reason for failure for up to n — 1 failed transmissions, its own status 
reports for up to n — 1 failed transmissions, the blacklist, list of eliminated nodes, and for each 
neighbor B G G, a list of nodes S G for which B knows the complete status report (see 
item 4 on line ([IHllS)). 

1. End of Transmission Parcel Qj. Modifications to this occur only on lines ([141 1581. 
(I15L 171). and (I15L 199). Every transmission, the inforgibility of the signature scheme 
and the comment on line (I14L 156) guarantee that the sender will add Qj to DB as on 
(I14L 158) at most once. Meanwhile, for every transmission, either (I15L 171) or (I15L 199) 
will be reached exactly once. Therefore, there is at most one End of Transmission parcel 
in DB at any time. 

2. Blacklist BL. We show that BL consists of at most n parcels at any time. More 
specifically, we will show that BL lives in the domain [l..n] x {0, 1}'^('°S")^ i.e. an array 
with n slots indexed by each G G, with each slot holding _L (if the corresponding 
node is not on the blacklist) or the index of the transmission in which the corresponding 
node was most recently added to the blacklist. To see this, notice that modifications to 
the blacklist occur only on lines (I14L 166). (|15L 171). and (|151 188). "Removing" a node 
A^ from BL as on (|141 166) means changing the entry indexed by A^ to _L. "Clearing" 
the blacklist as on (|15L 171) means making every entry of the array equal to _L. Finally, 
"adding" a node to the blacklist as on (I15L 188) means switching the entry indexed by A^ 
to be the index of the current transmission. 

3. Status Report Parcels. Modifications to this occur only on lines ([T4l l62) and 
([111171). We show in Lemma HOj below that for any node N e'P\S, DB will hold at 
most n(n — 1) status report parcels from A^ at any time, from which we conclude that 
DB need hold at most n(n — 1)^ status report parcels. 
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4. Participating Lists. We will view the participating list corresponding to transmission 
T as an array [l..n] x {0,1}^, where the array is indexed by the nodes, and an entry 
corresponding to node € G is either the index of the transmission T (if N participated 
in T) or _L otherwise. Therefore, since each participating list consists of n parcels, we can 
argue that participating lists require at most n{n — 1) parcels if we can show that DB 
need hold at most n — 1 participating lists at any time. To see this, notice that (I15L 187) 
is reached only in the case the transmission failed (|151 185). and we showed in Lemma 
110.41 that there can be at most n — 1 failed transmissions before a node is necessarily 
eliminated and DB is cleared as on (|151 171). 

5. Reason Transmissions Failed. Modifications to this occur only on lines ([151171), 
(HSligi), ([151194), and ([111197). Notice that of the latter three, exactly one will be 
reached if and only if the transmission failed. Also, each one of the three will add at 
most one parcel to DB. Since DB is cleared any time Eliminate Node is called as on 
(I15L 171). we again use Lemma [10.41 to conclude that Reason for Transmission Failures 
require at most n — 1 parcels of DB. 

6. Sender's Own Status Reports. These are added to DB on lines (fT5ll911. ([T5l l94). 
and (I15L 197). and removed from DB on (I15L 171). Notice that of the former three lines, 
exactly one will be reached if and only if the transmission failed. Also, each one of 
the three will add at most n parcels to DB. Since DB is cleared any time Eliminate 
Node is called as on (|15L 171). we again use Lemma [10.41 to conclude that Reason for 
Transmission Failures require at most n{n— 1) parcels of DB. 

7. List of Eliminated Nodes EN. Modifications to this occur only on line ([T5l l70l. 
Since EN is viewed as living in [l..n] x {0, l}0{i°g"), "adding" a node N to EN means 
changing the entry indexed by A^ from _L to the index of the current transmission. Notice 
that EN G [l..n\ x {0, l}0(i°g") can be expressed using n parcels. 

8. The Label of a Node A^ Whose Status Report is Known to B. Modifications 
to this occur only on lines ([I4ll60), ([141166), and ([151171). We show in LemmadLS] 
below that for any pair of nodes B, N G V \ S , DB will hold at most one parcel of the 
form (BjN,!') at any time (see e.g. (|141 160)). from which we conclude that DB need 
hold at most (n — 1)^ parcels of this type. 

Adding together these changes, the sender's DB needs to hold at most + + n parcels, 
as required. 

We have now shown each of the variables of Figures [9] and [10] have domains as indicated. ■ 

Lemma 11.3. For any node N ^ V \ S , the sender's data buffer will hold at most n{n — 1) status 
report parcels from N at any time. More specifically, let {Ti, . . . ,Tj} denote the set of transmissions 
for which the sender has at least one status report parcel from N . Then j < n — 1 and for every 
i < j, the sender has N 's complete status report for transmission Tj. 

Proof. We first note that the first sentence follows immediately from the latter two since each 
status report consists of at most n status report parcels (|141 142-145). Fix N G V \ S and let 
{Ti, . . . , Tj} be as in the lemma, ordered chronologically. We first show that j < n — 1. For the sake 
of contradiction, suppose j > n. We first argue that for all 1 < z < j, transmission Tj necessarily 
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failed. Fix 1 < i < j. Since DB contains a status report parcel from N for transmission Tj, it must 
have been added on (I14L 162) of some transmission T. Therefore, line (I14L 161) must have been 
satisfied, and in particular, (A^, Tj) must have been on BL during T. Therefore, (A^, Tj) must have 
been added to BL as on (I15L 188) of transmission Tj, which in turn implies transmission Tj failed 
(|151 185). Therefore, transmission Tj failed for each I < i < j. 

By Lemma 110.41 there can be at most n — 1 failed transmissions before a node is eliminated 
as on (|15L 169-177). Since j > n, considering failed transmissions {T2,...,Tj}, there must have 
been a transmission T2 < T < Tj such that Eliminate N (|151 169) was entered in transmission T. 
We first argue that T < Tj as follows. If T = Tj, then (A^,Tj) would not be added to BL as on 
(|151 188) (once the protocol enters (|15L 169). it halts until the end of the transmission (|151 177). 
thus skipping (|151 188)). But then (|141 161) of any transmission after Tj cannot be satisfied for any 
status report parcel corresponding to Tj, and hence none of A^'s status report parcels corresponding 
to Tj could be added to DB after transmission Tj. Similarly, none of A'''s status report parcels 
corresponding to Tj can be added to DB before or during transmission Tj by Claim 111.41 below. 
This then contradicts the fact that at some point in time, DB contains one of A^'s status report 
parcels corresponding to Tj. 

We now have that for some transmission Ti < T < Tj, Eliminate N is entered during T. 
Therefore, all of A^'s status report parcels for Ti are removed from DB on (1151 171) and (A^, Ti) is 
removed from BL on (1151 171) of transmission T < Tj. Since Ti < T, (A^, Ti) will never be put on 
BL as on (|151 190) for any transmission after T, and consequently, (|141 161) will never be satisfied 
after T for any of A^'s status report parcels from Ti. Therefore, none of A^'s status report parcels will 
be put into DB after they are removed on (I15L 171) of T. Meanwhile, by the end of transmission 
T < Tj, DB cannot have any of A^'s status report parcels corresponding to Tj by Claim [TL4l below. 
We have now contradicted the assumption that DB simultaneously holds some of A^'s status report 
parcels from Ti and Tj. Thus, j < n — 1, as desired. 

We now show that for every i < j, the sender has A^'s complete status report for transmission Tj. 
If j = 1, there is nothing to prove. So let 1 < j < n — 1, and for the sake of contradiction suppose 
there is some i < j such that the sender has at least one of A^'s status report parcels for Tj, but not 
the entire report. Let Tj denote the transmission that the status report parcel corresponding to Tj 
was added to DB as on (I14L 162). and let Tj+i denote the transmission that the parcel corresponding 
to Tj+i was added to DB as on (I14L 162). Without loss of generality, we suppose that Tj < Tj+i. 
Since (I14L 162) is entered during transmission Tj, it must be that (I14L 161) was satisfied, and in 
particular (A^, Tj) was on BL during Tj. Similarly, (A^, Tj+i) was on BL during Tj+i. Lemma [11.61 
below states that for each A^ € G, A^ is on BL at most once, i.e. there is at most one entry of the 
form (A^, T) on BL at any time. Since nodes are only added to BL at the very end each transmission 
(|151 188). we may conclude that we have strict inequality: Tj < Tj+i. In particular, (A^, Tj+i) was 
not on BL at the start of Tj, but (A^, Tj) was. Therefore, (A^, Tj+i) was added to BL as on (I15L 188) 
of transmission Tj+i, and so N € Vj (1151 187-188). In particular, A^ is not blacklisted by the end 
of Tj_|_i (I15L 187). Therefore, there must be some transmission T S |Tj,Tj+i] such that (A^,Tj) is 
removed from BL as on (|141 166) or (I15L 171). Both of these lead to a contradiction, as the first 
implies the sender has A^'s complete status report for Tj, while the latter implies that all status 
reports corresponding to Tj should have been removed from DB. ■ 

Claim 11.4. For any N ^ G and for any transmission 1, the sender's data buffer DB will never 
hold any of N 's status report parcels corresponding to T before or during transmission T. 
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Proof. Let N G, and for the sake of contradiction, let T be a transmission such that DB has 
one of A^'s status report parcels from T before or during T. Since status reports are only added to 
DB on (|14L 162). this implies that there is some transmission T' < T such that (|141 161) is satisfied 
at some point of T' before the Prepare Start of Transmission Broadcast of transmission T' is 
called. This in turn implies that {N, T) was on BL before the Prepare Start of Transmission 
Broadcast of transmission t' < T was called. However, this contradicts the fact that the only 
time (A^,T) can be added to BL is during the Prepare Start of Transmission Broadcast of 
transmission T on (I15L 188). ■ 

Claim 11.5. For any pair of nodes B,N G G\S, the sender's data buffer will hold at most one 
status report parcels of the form {B,N,T') at any time. 

Proof. Fix B,N G G\S, and suppose for the sake of contradiction that there are two transmissions 
T' and T" such that both {B,N,T') and {B,Nj") are in DB at the same time (note that T' / T" 
by the comment on I14L 156). Since parcels of this form are only added to DB on (|14L 160). we 
suppose without loss of generality that T is a transmission and t is a round in T such that {B, N, T") 
is already in DB when {B,Nj') is added to DBjis on ([I4ll60) of round t. Since ^160) is 
reached, (|141 159) was satisfied, so in particular {N,T') is on the sender's (current version of the) 
blacklist. Similarly, since (BjN,!") was (most recently) added to DB as on (I14L 160) of some 
round t of some transmission T < T, it must have been that {N, T") was on the sender's (version of 
the) blacklist during round t of T. By Lemma [11.61 below, since [N,!') is on the sender's current 
blacklist (as of round t of transmission T), and (A^,T") was on an earlier version of the sender's 
blacklist, it must be that {N, T") was removed from the blacklist at some point between round t of T 
and round t of T. Notice that nodes are removed from the blacklist only on (|141 166) and (|151 171). 
However, in both of these cases, {B,N,T") should have been removed from DB (see (|14L 166) and 
(|151 171)). contradicting the fact that it is still in DB when (S, A^, T') is added to DB in round t 
of transmission T. ■ 

Lemma 11.6. A node is on at most one blacklist at a time. In other words, whenever a node {N,T) 
is added to the sender's blacklist as on 1151 188 ). we have that {N,T') ^ BL for any other (earlier) 
transmission T'. Additionally, if {N,T') € BL at any time, then: 

1. Transmission T' failed 

2. No node has been eliminated since T' to the current time 

3. The sender has not received N 's complete status report corresponding to T' 

Proof. The first statement of the lemma is immediate, since the only place a (node, transmission) 
pair is added to BL is on (I15L 188). and by the previous line, necessarily any such node is not 
already on the blacklist. Also, Statement 1) is immediate since (I15L 188) is only reached if the 
transmission fails (I14L 185). To prove Statements 2) and 3), notice that (A^, T') is only added to 
the blacklist at the very end of transmission T' (|15L 188). In particular, if (A^, T') is ever removed 
from the blacklist during some transmission lEll as on (I14L 166) or (I15L 171). then {N,T') can never 
again appear on the blacklist (as remarked in the footnote, T > T', and so at any point during or 

^''If (A'', T') is removed from the blacklist as on lfT4ll66l or ifTsl lTll of transmission T, then necessarily T > T', 
since (N,T') can only be added to BL at the very end of a transmission I|15l l88l. i.e. lines I|14l l66l and I|141 1711 
cannot be reached after line I|15l l88l in the same transmission. 
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after transmission T, (A^, T') can never again be added to BL as on (I15L 188) since T' has already 
passed). Therefore, if during transmission T a node is ehminated as on (|151 169-177) or the sender 
receives A'^'s complete status report of transmission T' as on (|14L 164). then N will be removed from 
the blacklist as on (I14L 166) or (I15L 171). at which point (A^, T') can never be added to BL again 
(since necessarily T > T' as remarked in the footnote). This proves Statements 2) and 3). ■ 

Lemma 11.7. For any {N,J) on the sender's blacklist, the sender needs at most n parcels from N 
in order to have N 's complete status report, and subsequently remove N from the blacklist ( 1141 164- 

165;. 

Proof. This was proven when discussing the appropriateness of variable domains in Lemma ril.2i ■ 

We set the following notation for the remainder of the section. T will denote a transmission, 
G-Y will denote the set of non-eliminated nodes at the start of T, Vj will denote the participating 
list for T, and Hj will denote the uncorrupted nodes in the network. If the transmission is clear or 
unimportant, we suppress the subscripts for convenience, writing instead GjV, and Ti. 

Lemma 11.8. For any honest node A £ G and any transmission T, A must receive the complete 
Start of Transmission (SOT) broadcast before it transfers or re-shuffles any packets. Additionally, 
the signature buffers SIGa,a and SIG^ of any honest node A £ G are always cleared upon receipt 
of the complete SOT broadcast (and hence before any packets are transferred to / from/within A). 

Proof. Fix an honest node A £ G and a transmission T. If A has not received the full Start of 
Transmission (SOT) broadcast for T yet, then A will not transfer any packets (|121 59). (|lll 31-33). 
([12163) and ([11135-37). This means that ([12163) will always be satisfied, and hence ([12178) 
can never be reached, and so RR will remain equal to —1 ( (I10L 50). and (I15L 209)) so long as no 
codeword packets have been transferred. This in turn implies (I12L 46) cannot be satisfied before 
any codeword packets have been transferred. Putting these facts together, the signature buffers 
cannot change as on ([12148-50), ([12174-75), ([12180), or ([12182) before A receives the complete 
50 T broadcast. Also, no packets will be re-shuffled during the call to Re-Shuffle if no packets have 
moved during the Routing Phase, as the condition statement on ([3.74) was eventually false in the 
last round of the previous transmission, and the state of the buffers will not have changed if no 
packets have been transferred in the current transmission. Therefore, before A has received the 
complete 50 T broadcast, no packet movement to/from/within A is possible, and changes to the 
signature buffers are confined to the ones appearing on lines (I14L 128). (|141 133). (I14L 141). and 
(|141 146). all of which clear the signature buffers. 

Suppose now that A has received the full SOT broadcast for T. Recall that part of the SOT 
broadcast contains JIt = i\EN\,\BL\, F,*), where EN refers to the eliminated nodes, BL is the 
sender's current blacklist, F is the number of failed transmissions since the last node was eliminated, 
and the last coordinate denotes the reason for failure of the previous transmission (in the case it 
failed) ([111200). If \BL\ = 0, then A will clear all its entries of and SIGa,a on ([T1128). 

Otherwise, \BL\ > 0, and N will clear all its entries of SIG^ and SIGa,a when it learns the last 
blacklisted node on (I14L 146). Therefore, in all cases A^s signature buffers are cleared by the time 
it receives the full 50 T broadcast, and in particular before it transfers any packets in transmission 
T. ■ 

In order to prove a variant of Lemma [6.13l in terms of the variables used in the (node-controlling 
-|- edge-scheduling) adversary protocol, we will need to first re-state and prove variants of Lemmas 
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16.111 17.141 and 17.151 We begin with a variant of Lemma 16.111 (the first 5 Statements correspond 
directly with Lemma [6.111 the others do not, but will be needed later): 

Lemma 11.9. For any honest node A € G and at all times of any transmission: 

1. For incoming edge E[S,A), all changes to SIG'^lSjs^A are strictly non-negative. In particular, 
at all times: 

< SIG'^[3]s,A (29) 

2. For outgoing edge E{A,R), all changes to SIG^[3]a,r are strictly non-negativ^. In partic- 
ular, at all times: 

< SIG^[3]a,r (30) 

3. For outgoing edges E{A,B), B ^ R, all changes to the quantity {SIG^[3]a,b — SIG'^[2]a,b) 
are strictly non-negative. This remains true even if B is corrupt. In particular, at all times: 

< J] {SIG^[3]a,b-SIG^[2\a,b) (31) 

BeV\{A,S} 

4. For incoming edges E{B,A), B ^ S, all changes to the quantity {SIG'^[2]b,a — SIG^[3]b,a) 
are strictly non-negative. This remains true even if B is corrupt. In particular, at all times: 

< {SIG''[2]b,a-SIG^[3]b,a) (32) 

BeV\{A,S} 

5. All changes to SIGa,a o,re strictly non-negative. In particular, at all times: 

< SIGa,a (33) 

6. The net decrease in potential at A (due to transferring packets out of A and reshuffling 
packets within A's buffers) in any transmission is bounded by A's potential at the start of the 
transmission, plus A 's increase in potential caused by packets transferred into A. In particular: 

SIGa,a + SIG^[3]a,b < {4n'-6n^)+ ^ SIG^[3]b,a (34) 

BeV\A BeV\A 

7. The number of packets transferred out of A in any transmission must be at least as much as 
the number of packets transferred into A during the transmission minus the capacity of A 's 
buffers. In particular: 

4n^-8n> ^ {SIG^[1]b,a - SIG^[1]a,b) (35) 
BeV\A 



^^SIG^[3] along out going edges measures the decrease in potential as a positive quantity. Thus, a positive value 
for SIG^ [3] along an outgoing edge corresponds to a decrease in non-duplicated potential. 
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8. The number of times a packet p corresponding to the current codeword has been transferred 
out of A during any transmission is bounded by the number of times that packet has been 
transferred into A. In particula^^: 



Proof. We prove each inequality separately, using an inductive type argument on a node ^'s signa- 
ture buffers. First, note that all signature buffers are cleared at the outset of the protocol (|101 46). 
(|101 48). and (I10L 54). Also, anytime the signature buffers are cleared as on (|141 128). (|141 133). 
(|141 141). and (|141 146). then all of the statements (except possible Statement 8, which depends on 
values from potentially corrupt nodes B € G) will be true. So it remains to check the other places 
signature buffers can change values (([12148-50), ([12174-75), ([12180), ([12182), and ([376)), and 
argue inductively that all such changes will preserve the inequalities of Statements 1-7 (Statement 
8 will be proven separately). Since all of these lines represent packet movement, they can only be 
reached if A has received the complete 50 T broadcast for the current transmission (Lemma lll.811 . 
and so we may (and do) assume this is the case in each item below. In particular. Lemma 111.81 
states that because we are assuming A has received the complete SOT broadcast for transmission 
T, all of A's signature buffers will be cleared before any changes are made to them. 

1. Aside from being cleared, in which case ([29]) is trivially true, the only changes made to 
SIG^[3]s,A occur on (I12L 75). where it is clear that all changes are non-negative since Hop 
is non-negative (Statement 9 of Lemma [TTT] together with Lemma flLip . 

2. Aside from being cleared, in which case ([301) is trivially true, the only changes made to 
SIG^[3]a,r occur on (|12L 49). where it is clear that all changes are non-negative since Hpp 
is non-negative (Statement 9 of Lemma [TTT] together with Lemma flLip . 

3. Fix B £ V \ S, A. Intuitively, this inequality means that considering directed edge E{A,B), 
the decrease in A's potential caused by packet transfers must be greater than or equal to S's 
increase, which is a consequence of Lemma 16.111 Formally, we will track all changes to the 
relevant values in the pseudo-code and argue that at all times and for any fixed B £ G (honest 
or corrupt), if A is honest, then < SIG^[3]a,b — SIG'^[2]a,b- All changes to these values 
(aside from being cleared) occur only on (|12L 48-49) since here we are considering A^s values 
along outgoing edge E{A,B). Notice that Hpp cannot change between (I11L 08) of some round 
and (I12L 49) of the same round. Since lines (1121 48-49) are only reached if Verify Signature 
Two accepts the signature (otherwise RR is set to _L on (|121 90) and hence (|121 45) will fail), we 
have that SIG^[2]a,b changes by at most the value that Hpp had on (|121 89) (see comments 
on line (|12l 88-90)). and this is the value sent /received on lines (I11L 07) and (|lll ll) and 
eventually stored on (|12L 48). Meanwhile, when SIG^[3]a,b changes, for honest nodes it will 
always be an increase of Hpp (I12L 49). and as noted above, this value of Hpp is the same as it 
had on (I12L 89). Therefore, for honest nodes, whenever the relevant values change on (|121 48- 
49), the change will respect the inequality SIG^[3]a,b — SIG^[2]a,b > Hpp — Hpp = 0. 

^^Notice that ifSGjl is the only statement of the Lemma that involves quantities in the neighbors' signature buffers 
(in addition to A's buffers). Since there is no assumption made about the honesty of the neighbor's of A, this may 
seem problematic. However, we show in the proof that regardless of the honesty of A's neighbors -B € G, l|36p will be 
satisfied if A in honest. 




(36) 
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4. Fix B £ V \ S, A. Intuitively, this inequality means that considering directed edge E{B,A), 
the decrease in B's potential caused by packet transfers must be greater than or equal to ^'s 
increase, which is a consequence of Lemma 16.111 Formally, we will track all changes to the 
relevant values in the pseudo-code and argue that at all times and for any fixed B G G (honest 
or corrupt), if A is honest, then < SIG^[2]b,a — SIG"^[3]b,a- All changes to these values 
(aside from being cleared) occur only on (|12l 74-75) since here we are considering A^s values 
along incoming edge E{B,A). When SIG^[2]b,a chan ges on (|12L 74). they take on the values 
sent by B on (I12L 60) and received by A on (|121 62). However, in order to reach (|121 74). 
the call to Verify Signature One on (I12L 69) must have returned true. In particular, the 
comments on (1121 84-86) require that A verify that the change in SIG^[2]b,a that B sent to A 
is at least Hop bigger than the previous value A had from B. Meanwhile, when SIG^[3]b,a 
changes, for honest nodes it will always be an increase of Hqp (|121 75). Therefore, since 
Hgp cannot change between (I12L 84) of some round and (I12L 75) later in the same round for 
honest nodes, whenever the relevant values change on (1121 74-75). the change will respect the 
inequality SIG^[2]b,a - 5/G^[3]b,a > Hgp - Hgp = 0. 

5. Intuitively, this inequality says that all changes in potential due to packet re-shuffling should 
be strictly non-positive {SIGa,a measures potential drop as a positive quantity), which is a 
consequence of Lemma l6.111 Formally, all changes made to SIGa,a (aside from being cleared) 
occur on ([71.76), where the change is M -|- m — 1. The fact that this quantity is strictly non- 
negative for honest nodes follows from Claim 16.41 

6. Since the inequality concerns SIGa,a and SIG[3] (along both incoming and outgoing edges), 
we will focus on changes to these values when a packet is transferred (or re-shuffled). More 
specifically, we will look at a specific packet p and consider p's affect on A^s potential during 
each of p's stays in A, where a "stay" refers to the time A receives (an instance of) p as on 
(I12L 77) to the time it sends and gets confirmation of receipt (as in Definition 17. 6p for (that 
instance of) We fix p and distinguish between the four possible ways p can "stay" in A: 

(a) The stay is initiated by A receiving p during T and then sending p at some later round of 
T, and getting confirmation of p's receipt as in Definition 17.61 More specifically, the stay 
includes an increase to some incoming signature buffer SIG^[3] as on (|121 75) and then 
an increase to some outgoing signature buffer SIG^[3] as on (I12L 49). Let B denote the 
edge along which A received p in this stay, and B' denote the edge along which A sent p. 
Then SIG^[3]b,a will increase by Hgp on (I12L 75) when p is accepted. Let M denote the 
value of Hgp when p is received. The packet p is eventually re-shuffled to the outgoing 
buffer along E{A,B'). Let m denote the value of Hpp when (|121 49) is reached, so that 
the change to SIG^[3\a,b' due to sending p is ra. By Statement 3 of Claim ITTTI (which 
remains valid by Lemma flLip . any packet that is eventually deleted as on (1121 50-51) 
will be the fiagged packet, and so the packet that is deleted did actually have height m 
in A's outgoing buffer. In particular, the packet began its stay in an incoming buffer 
at height M, and was eventually deleted when it had height m in some outgoing buffer. 
In particular, since SIGa,a accurately tracks changes in potential due to re-shuffling 
(Statement 1 of Lemma ril.l6p . we have that during this stay of p, SIGa,a changed by 

"a given packet p may have multiple stays in A during a single transmission, one for each time A sees p. 
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M — m. Therefore, considering only p's affect on the following terms, we have that: 

SIGa.a + SIG^^A^B' - SIG^[i\B,A = {M-m) + m-M = (37) 

(b) The stay begins at the outset of the protocol, i.e. p started the transmission in one of A's 
buffers, and the stay ends when p is deleted (after having been sent across an edge) in sonne 
round of T. More specifically, there is no incoming signature buffer SIG'^lS] that changes 
value as on (|12L 75) due to this stay of p, but there is an increase to some outgoing 
signature buffer 5IG'^[3] as on (|12L 49). Using the notation from (a) above with the 
exception that M denotes the initial height of p in one of ^'s buffers at the start of T, 
then considering only p's affect on the following terms, we have that: 

SIGa,a + SIG^[3]a,b' = {M -m) + m = M (38) 

(c) The stay is initiated by A receiving p during T, but p then remains in A through the end of the 
transmission (either as a normal or a flagged packet). More specifically, the stay includes 
an increase to some incoming signature buffer SIG^[3] as on (I12L 75). but there is no 
outgoing signature buffer 5/G'^[3] that changes value as on (I12L 49) due to this stay of 
p. Using the notation from (a) above with the exception that m denotes the final height 
of p in one of ^'s buffers at the end of T, then considering only p's affect on the following 
terms, we have that: 

SIGa,a - SIG'^[3]b,a = {M -m) - M = -m<0 (39) 

(d) The stay begins at the outset of the protocol, i.e. p started the transmission in one of A's 
buffers, and p remains in A's buffers through the end of the transmission (either as a normal 
or a flagged packet). More specifically, there is no incoming signature buffer 5/G'^[3] 
that changes value as on (1121 75) due to this stay of p, and there is no outgoing signature 
buffer SIG^[3] that changes value as on (I12L 49) due to this stay of p. Letting M denote 
the initial height of p in one of A's buffers at the start of T and m the final height of p in 
one of A^s buffers at the end of T, then considering only p's affect on the following terms, 
we have that: 

SIGa,a = M-m<M (40) 

We note that the above four cases cover all possibilities by Claim 16.81 (which remains valid 
since A is honest, and Lemma m.ip . We will now bound SIGa,a + ^BeP\A 'S^C!^['^]a,b — 
SIG^[3]b,a by adding all contributions to SIGa,a and 5/G^[3]a,b' and SIG^[3]b,a from 
all stays of all packets and for all adjacent nodes B,B'. Notice that ignoring contributions 
as in Case (c) will only help our desired equality, and contributions as in Case (a) are zero, 
so we consider only packet stays as in ([38]l and ([40]l . Since these contributions to potential 
correspond to the initial height the packet had in one of A^s buffers at the outset of T, the 
sum over all such contributions cannot exceed A^s potential at the outset of T, which for an 
honest node A is bounded by 2(n — 2)2n(2n + l)/2 < — 6n^ (see e.g. proof of Claim [621) ■ 

7. Intuitively, this inequality means that because a node can hold at most 2(n — 2)(2n) packets 
at any time, the difference between the number of packets received and the number of packets 
sent by an honest node will be bounded by 4n^ — 8n. Formally, during a transmission T, the 
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only places the quantities change are on (I12L 74) and (I12L 48). As with the proof of 

Statement 6 above, we consider the contribution of each packet p's stay in 



(a) The stay is initiated by A receiving p during T and then sending p at some later round of 
T, and getting confirmation of p's receipt as in Definition 17.61 More specifically, the stay 
includes an increase to some incoming sig nature buffer SIG^[l] as on (|121 74) and then 
an increase to some outgoing signature buffer S'/G'^[l] as on (|121 48). Let B denote 
the edge along which A received p in this stay, and B' denote the edge along which A 
sent p. Since A will be verifying that B (respectively B') signed the correct values (see 
comments on ^84-86) and ([l2l88-90)), we have that SIG'^[1]b,a will increase by 1 
on (|12L 74) due to receiving p for the first time, and SIG^[1]a^b' will increase by 1 when 
it receives confirmation of receipt for sending p as on (|121 48). Therefore, considering 
only p's affect on the following terms, we have that: 

SIG^[1]b,a - SIG^[1]a,b' = 1-1 = (41) 

(b) The stay is initiated by A receiving p during T, but p then remains in A through the end of the 
transmission (either as a normal or a flagged packet). More specifically, the stay includes 
an increase to some incoming signature buffer 5/G'^[l] as on (112L 74). but there is no 
outgoing signature buffer ^/^^[l] that changes value as on (I12L 48) due to this stay of 
p. Using the notation from (a) above, then considering only p's affect on the following 
terms, we have that: 

SIG^[1]b,a = 1 (42) 

We note that the above two cases cover all possibilities by Claim [6^ (which remains valid since 
A is honest, see Lemma fll-ip . We now add all contributions to SIG^[1]a,b' and SIG^[1]b,a 
from all stays of all packets from all neighbors. Notice that the only non-zero contributions 
come from packets stays as in ([42]) . and these contributions will correspond to packets that 
are still in A's buffers at the end of the transmission. Since an honest node A can end the 
transmission with at most 2(n — 2)(2n) packets, summing over all such contributions results 
cannot exceed 4n^ — 8n, as required. 

8. Intuitively, this is saying that an honest node cannot output a packet more times than it 
inputs the packet (see Claim 16. Sp . Note that this is the only place in the theorem that 
depends on status reports not originating from A {SIG^[p] is a status report parcel from 
B). A priori, there is the danger that a corrupt B can return a faulty status report, thereby 
framing A. However, because SIG^[p]a,b includes a valid signature from A, the inforgibility 
of the signature scheme guarantees that the only way a corrupt node B can frame A in this 
manner is by reporting out-dated signatures. But if A is honest, then SIG^[p]a,b is strictly 
increasing in value as the transmission progresses (the only place it changes is (I12L 74). which 
comes from the value received on (|121 62). corresponding to the value sent on (|121 60)). and 
hence a corrupt B cannot "frame" A by reporting outdated signatures for SIG^[p]a,b', indeed 
such a course of action only helps the inequality stated in the theorem. Also notice that 

^^Note that necessarily p is a packet corresponding to the current codeword, since packets corresponding to old 
codewords do not increment SIG[1], see comments on I|lll 59-60l and Therefore, there are only two cases 

to consider. 
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(other than out-dated signatures) the only place B gets valid signatures from A is on (|121 62). 
and this value is one higher than the value that A itself is recording (I12L 60) until A updates 
SIG^\p]a,b on (|12L 48). We argue in case (b) below, that whenever B has received an updated 
SIG^\p\a,b as on ((12174) but A has not yet updated SIG^\p\a,b as on (fT2l48l (and so these 
two values differ by one), then Case (b) will contribute -1 to the sum in (f36]l . and therefore 
the difference of +1 between SIG^\p]a,b and SIG^\p]a,b will exactly cancel. These two facts 
allow us to argue (f36]l by using SIG^[p]a,b instead of SIG^[p]a,b- 

Formally, during a transmission T, the only places the quantities SIG\p] change are on (I12L 74) 
and (I12L 48). As with the proof of Statement 3 above, we consider the contribution of each 
packet p's stay in yj^: 

(a) The stay is initiated by A receiving p during T and then sending p at some later round of 
T, and getting confirmation of p's receipt as in Definition 17.61 More specifically, the stay 
includes an increase to some incoming signature buffer SIG^\p] as on (|121 74) and then 
an increase to some outgoing signature buffer SIG^\p] as on (|121 48). Let B denote 
the edge along which A received p in this stay, and B' denote the edge along which A 
sent p. Since A will be verifying that B (respectively B') signed the correct values (see 
comments on ([l2l84-86) and ([12188-90)), we have that SIG'^[p]b,a will increase by 1 
on (|121 74) due to receiving p for the first time, and SIG^[p]a,b' will increase by 1 when 
it receives confirmation of receipt for sending p as on (|121 48). Therefore, considering 
only p's affect on the following terms, we have that: 

SIG^[p]a,b' - SIG^[p]b,a = 1-1 = (43) 

(b) The stay is initiated by A receiving p during T, but p then remains in A through the end of the 
transmission (either as a normal or a flagged packet). More specifically, the stay includes 
an increase to some incoming signature buffer SIG^\p] as on (I12L 74). but there is no 
outgoing signature buffer SIG'^lp] that changes value as on (I12L 48) due to this stay of 
p. Using the notation from (a) above, then considering only p's affect on the following 
terms, we have that: 

- SIG^[1]b,a = -1 (44) 

We note that the above two cases cover all possibilities by Claim [6^ (which remains valid since 
A is honest, see Lemma flLip . We now add all contributions to SIG^[p]a,b' and SIG^\p]b,a 
from all stays of p from all neighbors on V (note that it is enough to consider only neighbors 
on V by Claim [Tl.lSp . Notice that (l43]) does not contribute anything, so we have that: 

^ (5/G^[p]a,b - SIG'^[p]b,a) = -X, (45) 
BeV 

where x is the number of times Case (b) occurs. Notice that ([361) is interested in SIG^\p]a,b 
(as opposed to SIG^[p]a,b)- However, since B cannot report values of SIG^[p]a,b from 

■^^Note that necessarily p is a packet corresponding to the current codeword, since packets corresponding to old 
codewords do not increment SIG[p], see comments on 11121 59-601 and Therefore, there are only two cases 

to consider. 
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previous transmissiona I. the only inaccurate value that B can report in its status report 
parcel concerning SIG^[p]a,b is by using an older value from T. As discussed above, cheating 
in this manner only serves to help (f36l) . On the other hand, if B does report the valid 
value for SIG^[p]a,b (i-e. not outdated), then Lemma [11.171 guarantees that SIG^[p]a,b — 
SIG^[p]a,b < 1, with equality if SIG^\p]a,b has been updated as on (I12L 74) and SIG^[p]a,b 
has not yet been updated after this point as on (|121 48). Notice that every time this happens, 
we fall under Case (b) above, and in particular it can happen at most x times (see definition 
of X above). Therefore: 

^ (5/G^[p]a,b - SIG^[p]b,a) <x+J2 (SIG'^IpUb - SIG^[p]b,a) =x-x = 0, (46) 

BeV B&V 
which is (f36l) . 

All Statements of the Theorem have now been proven. ■ 
We now prove a variant of Lemma 17.141 

Lemma 11.10. Suppose that A, B & G are both honest nodes, and that in round t, B accepts (as 
in Definition \6. 5]) a packet from A. Let Oa,b denote A's outgoing buffer along E{A,B), and let H 
denote the height the packet had in Oa,b when Send Packet was called in round t ( 1111 20 ). Also 
let Ib,a denote B's incoming buffer along E{A,B), and let I denote the height of Ib,a at the start 
oft. Let AifB denote the change in potential caused by this packet transfer, from B's perspective. 
More specifically, define: 

ipB := SIG^ [2]a,b - SIG^ [3]^,^ (47) 

and then IS.lpb measures the difference between the value of (pB at the end oft and the start oft. 
Then: 

AipB>H-I-l OR A^B>H (ifB = R) (48) 

Furthermore, after the packet transfer but before re-shuffling, Ib,a will have height /+ 1. 

Proof. By definition, B accepts the packet in round t means that (I12L 77) was reached in round 
t, and hence so was ([12174-75). In particular, SIG^[3]a,b will increase by Hqp on (I12L 75) (if 
B = R, then SIG^[3]a,b will not change on this line- see comment there). By Statements 1 and 
2 of Lemma ITTT] (which remain valid since B is honest by Lemma [11.1(1 . Hop < I + and hence 
SIG^[3]a,b will increase by at most I + 1. Also, since B had height / at the start of the round, 
and B accepts a packet on (I12L 77) of round t, B will have I + 1 packets in / when the re-shuffling 
phase of round t begins, which is the second statement of the lemma. 

Meanwhile, SIG^[2]a,b will change on (I12L 74) to whatever value B received on (|121 62) (as sent 
by A on (|12L 60) earlier in the round). Since A is honest, this value is Hpp larger than A's current 
value in SIG^[3]a,b (I12L 60). By Lemma [11.171 the value of SIG^[3]a,b at the start oft equals the 
value of SIG^[2]a^b at the start {before B has accepted the packet) of t. Therefore, the change in 
SIG^[2]a,b from the start of the round to the end of the round will be the value of Hpp = H when 
A reached (I12L 60) in round t (by definition of H and Statement 3 of Claim [72]). Since these are 
the only places SIG^[3\a,b and SIG^[2]a,b change, we have that Alps = H — Hgp > H — I — 1, 
as desired (if B = R, then AipB = H). ■ 

''^We are only interested in packets p corresponding to the current codeword, and all signatures that A provides 
for SIG^[p\a,b include the transmission index, so A^s honesty plus the inforgibility of the signature scheme imply 
that B cannot have any valid signatures from A contributing to SIG^[p\a,b before the current transmission T. 
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The following is a variant of Lemma 17.151 



Lemma 11.11. Let C = N1N2 ■ ■ ■ Ni be a path consisting of I honest nodes, such that R = Ni and 
S ^ C. Suppose that in some non-wasted round t, all edges E{Ni, Ni^i) , 1 <i < I are active for 
the entire round. For 1 < i < I , let A(f) denote the following changes to SIGNi,Ni and SIG^' during 
round t: 

1. Changes to (p^. (see notation of Lemma \11.10\ ). 

2. Changes to SIG^.^Ni 

Then if Oni,N2 denotes Ni 's outgoing buffer along E{Ni, N2), we have: 

- If On-^^n2 has a flagged packet that has already been accepted by N2 before round t, then: 

A(/) > O - / + 1 (49) 

- Otherwise, 

A(l)> 0-1 + 2 (50) 
where O denotes its height at the outset oft. 

Proof. Since A and B are honest, we use Lemma 111.11 and then follow exactly the proof of the 
analogous claim for the edge-scheduling model (Lemma l7.15p . In particular, the exact proof can be 
followed, using the fact that signature buffers record accurate changes in non-duplicated potential 
(Statement 1 of Lemma [11.16(1 . and using Lemma [11.91 in place of Lemma [6.111 and Lemma [11.101 
in place of Lemma 17.141 ■ 

Lemma 11.12. // at any point in any transmission 1, the number of blocked rounds is (3t, then 
the participating honest nodes of G will have recorded a drop in non- duplicated potential of at least 
n{/3j — 4n^). More specifically, the following inequality is true: 

n(/3T-4n3)< Y.SIGa,a + Y1 E {SIG'^[2]b,a - SIG^[3]b,a) (51) 
A&H\S AeH\S BeP\{A,S} 

Proof. For every blocked, non- wasted round t, by the conforming assumption there exists a chain Ct 
connecting the sender and receiver that satisfies the hypothesis of Lemma [ll.lll Letting A^i denote 
the first node on this chain (not including the sender), the fact that the round was blocked (and 
not wasted) means that A'^i's incoming buffer was full (see Lemma [11. ip . and then by Lemma [6?3l 
so was A^i's outgoing buffer along E{Ni, N2). Since the length of the chain I is necessarily less than 
or equal to n. Lemma [11.111 says that the change of (see notation there) in round t satisfies: 

A(/> > OATi.ATa - / + 1 > 2n - n-M > n (52) 

Since Acj) only records some of the changes to the signature buffers, we use Lemma 111.91 to argue 
that the contributions not counted will only help the bound since they are strictly non-negative. 
Since we are not double counting anywhere, each non-wasted, blocked round will correspond to an 
increase in Acj) of at least n, which then yields the lemma since the number of wasted rounds is 
bounded by 4n^ (Lemma ll0.9p . ■ 
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Lemma 11.13. If there exists A,B gG such that one of the following inequalities is not true, then 
either A or B is necessarily corrupt, and furthermore the sender can identify conclusively^ which 
is corrapQ; 

1. 5IG^[2]a,b < SIG^[^]a,b + 2n 

2. SIG^[2]s,A-SIG^[2]s,A<2n 

3. \SIG'^[1]b,a - SIG^[1]b,a\ < 1 and \SIG^[1]a,b - SIG^[1]a,b\ < 1 

(53) 

Proof. As in the first paragraph of the proof of Lemma ril.91 we may assume that both A and B have 
received the full Start of Transmission broadcast for T, so SIG^ and SIG^ should both be cleared 
(if A and B are both honest) of its values from the previous transmission before being updated with 
values corresponding to the current transmission T. We prove each Statement separately: 

1. That either A or i? is necessarily corrupt follows from Lemma [11.171 It remains to show 
that the sender can identify a node that is necessarily corrupt. We begin by assuming that 
SIG^\2]a,b and 5/G'^[3]a,b have appropriate signatures corresponding to T (otherwise, they 
either would not have been accepted as a valid status report parcel on (I14L 161). or a node will 
be eliminated as on ll4l l63). We now show that if the inequality in Statement 1 is noi true for 
some A,BgG, then A is necessarily corrupt. Notice that if A is honest, then SIG^[^]a,b is 
monotone increasing (other than being cleared upon receipt of the 50 T broadcast, SIG^[i]A,B 
is only updated on I12L 49). Similarly, other than being cleared upon receipt of the SOT 
broadcast, SIG^[2]a,b is only updated on (I12L 74). and tracing this backwards, this comes 
from the value received on (I12L 62) which in turn was sent on (|12L 60). Therefore, since B 
cannot forge ^'s signature (except with negligible probability or in the case A and B are 
both corrupt and colluding), SIG^[2\a^b can only take on values A sent B as on (|121 60). 
Meanwhile, as mentioned, if A is honest, SIG^[^]a,b is monotone increasing, and thus an 
honest A will never send a value for 5/G^[3]a,b on (I12L 60) of some round that is smaller 
than a value it sent for SIG^[2>\a,b on (I12L 60) of some earlier round. Therefore, since the 
value A is supposed to send B is Hpp < 2n (the inequality follows from Statement 9 of 
Lemma [7m and Lemma [11. ip . unless A is corrupt or B has broken the signature scheme, B 
will never have a sig ned value from A such that SIG'^[2]a,b > 2n + SIG^[3]a,b- Therefore, 
if the inequality in the first statement is not satisfied, A is necessarily corrupt (except with 
negligible probability) . 

2. That A is necessarily corrupt follows from Lemma fl 1.1 71 and the fact that the sender cannot 
be corrupted by the conforming restriction placed on the adversary. 

3. Note that the two statements are redundant, since the second is identical to the first after 
swapping the terms on the LHS and re-labelling. We therefore only consider the second 
inequality of Statement 3. That either ^ or S is necessarily corrupt follows from Lemma 
111.171 It remains to show that the sender can identify a node that is necessarily corrupt. As 

As long as the adversary does not break the signature scheme, which will happen with all but negligible probability, 
the sender will never falsely identify an honest node. 

""'The values of the quantities SIG^ and SIG^ all correspond to a common transmission T and refer to values the 
sender has received in the form of status reports for T as on I|141 1611. 
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in the proof of Statement 1 above, we begin by assuming that SIG^[1]a,b and SIG^[1]a,b 
have appropriate signatures corresponding to T (otherwise, they either would not have been 
accepted as a vahd status report parcel on (|141 161). or a node will be eliminated as on 
[11163). We now show that if \SIG'^[1]a,b - SIG^[1]a,b\ > 1 for some A,B e G, then 
either ^ or i3 is necessarily corrupt, and the sender can identify which one is corrupt. 

Notice that the quantities SIG^[1\a,b and SIG'^IIIa^b include the round in which the 
quantity last changed ( (IllL ll) and (I12L 60)). Let denote the round SIG^[1]a,b indicates it 
was last updated (which has been signed by A), and denote the round SIG'^[1]a,b indicates 
it was last updated (which has been signed by B. Note that these quantities refer to the values 
returned to the sender in the form of status report parcels, and node A (respectively B) has 
signed the entire parcel SIG'^[1]a,b (respectively SIG^[1]a,b), indicating this is indeed the 
parcel he wishes to commit to as his status report. We assume \SIG^[1]a,b — SIG^[1]a,b\ > 1, 
and break the proof into the following two cases: 

Case 1: tA > ts- We will show that B is corrupt. Notice that the fact that A has a 
valid signature on SIG^[1]a,b from B for round means that (with all but negligible 
probability that A could forge B's signature, or if A and B are both corrupt, allowing 
A to forge B's signature) B sent communication as on (IllL ll) of with the fifth 
coordinate equal to the value A used for SIG'^[1]a,b- In particular, this fifth coordinate 
represents the value B has stored for SIG^[1]a,b during tA- Since < t^, B does not 
update SIG^[1]a,b from through the end of T, and hence the value for SIG^[1]a,b 
that B returns the sender in its status report should be the same as the value B sent 
to A on (IllL ll) of round t^, which as noted above equals the value of SIG^[1]a,b 
that A returned in its status report. However, since this is not the case {SIG^[1]a,b 7^ 
SIG^[1]a,b), B has returned an outdated signature and must be corrupt. 

Case 2: t^i < t^. If = = 0, i.e. both nodes agree that they did not update their 
signature buffers along E{A, B) in the entire transmission (except to clear them when 
they received the 50 T broadcast), then necessarily both SIG'^[1]a,b and SIG^[1]a,b 
should be set to _L, so if one of them is not _L, the node signing the non-_L value can 
be eliminated. So assume that one of the nodes has a valid signature from the other 
for some round in T (i.e. that > 0). We will show that A is corrupt in a manner 
similar to showing B was corrupt above. Indeed, since B has a valid signature from 
A on SIG^[1]a,b from round t^, unless A and B are colluding or B has managed to 
forge ^'s signature, this value for SIG^[1\a,b comes from the communication sent by 
A on (|12L 60). In particular, since < and A claims he was not able to update 
SIG'^[1]a,b after round tA, the value A signed and sent on (I12L 60) should be exactly 
one 1 more than the value stored in SIG^[1]a,b as of line (|11L 07) of round t^, the latter 
of which was returned by A in its status report (by definition of t a and the inforgibility 
of the signature scheme) . But since \SIG^[1]a,b - SIG^[1]a,b\ > 1, this must not be 
the case, and hence A is corrupt. ■ 

Corollary 11.14. // there exists a node A £ G such that: 

4n3 - 4n2 < SIGa,a + Yl SIG''[2]a,b - SIG^[3]b,a, (54) 

BeV\A 
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then either a node can be eliminated as in Statement 1 of Lemma \11.13\ or as in Statement 6 of 
Lemma \11.9l 

Proof. Suppose no node can be eliminated because of Statement 1 of Lemma 111.131 so that for all 
B eG: 

SIG''[2]a,b < SIG^[3]a,b + 2n. (55) 
Then if (f54|l is true, we have that: 

4n3 - < SIGa,a + Yl SIG''[2]a,b - SIG^[3]b,a 

B(iV\A 

<5/GA,A+2n2+ ^ SIG^[2>]a,b-SIG^[S\b,a (56) 

B&V\A 

where the second inequality follows from applying ((55]) to each term of the sum. Therefore, A can 
be eliminated by Statement 6 of Lemma 111.91 ■ 

Corollary 11.15. In the case a transmission fails as in F2, the increase in network potential due 
to packet insertions is at most 2nD + 2n^. In other words, either there exists a node A ^ G such 
that the sender can eliminate A, or the following inequality is tru^^: 

^ SIG'^[3]s,A < 2nD + 2r? (57) 

A&V\S 

Proof. If the inequality in Statement 2 of Lemma 111.131 fails for any node ^4 E "P \ 5, the sender 
can immediately eliminate A. So assume that the inequality in Statement 2 of Lemma [11.131 holds 
for every ^ S "P \ 5". The corollary will be a consequence of the following observation: 

Observation. If a transmission T fails as in F2, then: 

^ SIG^[2]s,A<2nD (58) 

Aev\s 



Proof. Let Kj denote the value that k had at the end of T. Then formally, a transmission 
falling under F2 means that Kt is less than D. The structure of this proof will be to first 
show that for any A ^ V \ S , anytime SIG^[1\s^a is updated as on (|121 48). it will always 
be the case that 2n * SIG^[1\s^a > SIG^[2]s^a (so that in particular that the final value for 
SIG^[2\s^A at the end of T is less than or equal to 2n times the final value for SIG^[\]s,a)- 
We will then show that at the end of T: '}^Ae.'P\s S^G^[^]s,A = ^t- From these two facts, we 
will have shown: 

^ SIG^[2]s,A < ^ 2n * SIG^[l]s^A = 2nKj < 2nD (59) 
A£V\S AeV\S 

as required. 

""^The values of the quantities SIG^ correspond to some transmission T and refer to values the sender has received 
in the form of status reports for T as on I|141 1611. 
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The first fact is immediate, since for any A ^ V \ S , whenever SIG^[2]s^a is updated as on 
(I12L 48). the statement on (I12L 45) must have been satisfied, and so the statement on (I12L 89) 
must have been false. In particular, the change in SIG^[1]s,a was exactly one, and the change 
in SIG^[2]s,A was at most Hpp < 2n, where the inequality comes from Statement 9 of Lemma 
17.11 and Lemma 0.1.11 (see comments on lines (|12l 88-90)). The second fact is also immediate, 
as K and SIG^[1]s,a all start the transmission with value zero (or _L) by lines (I10L 54). (|101 70). 
(I15L 199). and (I15L 213). and then k is incremented by one on line (1121 47) of the outgoing 
buffer along some edge E{S, N) if and only if SIG^[1]s^n is incremented by one as on (I12L 48) 
(as already argued, changes to SIG^[1]s,a as on (112L 48) are always increments of one, see 
e.g. the comments on lines (|12l 88-90)). □ 

The corollary now follows immediately from the following string of inequalities: 

2nD> SIG'^[2]s,A 
Aev\s 

>-2n^+ SIG^[3]s,A 
Aep\s 

where the top inequality is the statement of the Observation and the second inequality comes from 
applying the inequality in Statement 2 of Lemma 01.131 to each term of the sum. ■ 

Lemma 11.16. For any honest node N €z G and for any transmission T; 

1. Upon receipt of the complete Start of Transmission (SOT) broadcast for transmission T, 
SIGn,n will be cleared. After this point through the end of transmission T, SIG]\f,N stores 
the correct value corresponding to the current transmission J (as listed on^l2). 

2. Suppose that N transfers at least one packet during T (i.e. N sends or receives at least one 
packet, as on ( 1121 60 ) or 1121 74-78 )). Then through all transmissions after! until the trans- 
mission and round (T',t' G T') that N next receives the complete SOT transmission for T', 
one of the following must happen: 

(a) All of N 's signature buffers contain information (i.e. signatures from neighbors) pertain- 
ing to T, or 

(b) All of N 's signature buffers are clear and N 's broadcast buffe¥^ contains all of the in- 
formation that was in the signature buffers at the end of T, or 

(c) {N,T,T') is not on the blacklist for transmission T' 

3. If N has received the full SOT broadcast for T, then all parcels in N 's broadcast buffer'^^BB 
corresponding to some node N 's status report are current and correct. More precisely: 

(a) If {N,T) is on the sender's blacklist, and at any time N has stored a parcel of N 's 
corresponding status report in its broadcast buffer BB, then this parcel will not be deleted 
until {N,T) is removed from the sender's blacklist. 

^^Oi the Data Buffer in the case N = S. 
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(b) If {N,T,T') is a part of the SOT broadcast of transmission T' , then upon receipt of this 
parcel, all of N 's status report parcels in N 's broadcast buffer correspond to transmission 
T' and are of the form as indicated on 1141 141-144 ). where the reason for failure of 
transmission T' was determined as on ( fl5L 190 ). 1151 193 ). or ) fl5l l96 ). 

4- If at any time N is storing a parcel of the form (5, A^, T) in its broadcast buffer (indicating B 
knows N 's complete status report for transmission T), then this will not be deleted until (A^,T) 
has been removed from the blacklist. 

Proof. Fix an honest N & G and a transmission T. We prove each Statement separately: 

1. The first part of statement 1 is Lemma [11.81 To prove the second part, we track all changes 
to SIGn,n and show that each change accurately records the value SIG^^n is supposed to 
hold. The only changes made to SIG^^jy after receiving the full SOT broadcast occur on 
lines ((376), dlSSO), ^80), and 1^82). Meanwhile, SIGn,n is supposed to track all 
packet movement that occurs within A^'s own buffers (i.e. all packet movement except packet 
transfers). The only places packets move within buffers of are on lines ([3.89-90), (|121 50). 
([I2l80), and 1^82). By the comments on lines (flSSO), ([I2l53), (HSlSO), and ^82), it is 
clear that SIGn,n appropriately tracks changes in potential due to the call to Fill Gap, while 
packet movement as on (I12L 53) does not need to change SIGn,n as packets are swapped, and 
so there is no net change in potential. In terms of re-shuffling ([389-90), we see that every 
packet that is re-shuffled causes a change in SIGn^n of M — m — 1 ([376). Notice the actual 
change in potential matches this amount, since a packet is removed from a buffer at height 
M ([390), reducing the height of that buffer from M to M — 1 (a drop in potential of M), 
and put into a buffer at height m + 1, increasing the height of the buffer from m to m + 1 (an 
increase of m + 1 to potential). 

2. If = S, there is nothing to show, since the sender's signature buffers' information is stored 
as needed on (I15L 191). (|151 194). and (I15L 197). and they are then cleared at the end of 
every transmission on (I15L 171) or (|151 199). For any N ^ S, we show that from the time A^ 
receives the full 50 T broadcast in a transmission T through the next transmission T' in which 
N next hears the full SOT broadcast, either all of N's signature buffers contain information 
from the last time they were updated in some round of T, or they are empty and either this 
information has already been transferred to A^'s broadcast buffer or A^ is not on the blacklist 
for transmission T' (this will prove Statement 2). During transmission T, there is nothing to 
show, as all changes made to any signature buffer over-write earlier changes, so throughout 
T, the signature buffers will always contain the most current information. It remains to show 
that between the end of T and the time A^ receives the full SOT broadcast of transmission T', 
the only change that A^'s signature buffers can make is to be cleared, and this can happen 
only if either the information contained in them is first transferred to A'^'s broadcast buffer, or 
if (A", T, T') does not appear in the S'OT broadcast of transmission T' (and hence the signature 
information will not be needed anyway). To do this, we list all places in the pseudo-code 
that call for a change to one of the signature buffers or removing data from the broadcast 
buffer, and argue that one of these two things must happen. In particular, the only places 
the signature buffers of A^ change (after initialization) are: (fT2l48-49). (fT2l50). ([T21 74-75). 
([12180), ([12182), ([11128), ([11133), ([11141), ([11146), and ([376). The only place that 
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information that was once in one of A^'s signature buffers is removed from the broadcast buffer 
is ([141134). 

First notice that because transfers a packet in transmission T, must have received the 
complete 50 T broadcast for transmission T (Lemma lll.Sp . For all rounds of all transmissions 
between T + 1 and the time N receives the full SOT broadcast for transmission T', lines 
([12148-51), ([12174-78), and ^76) will never be reached by N (see Lemma HLH] and its 
proof). Similarly, line (|121 80) will never be reached since (I12L 63) will always be satisfied. 
Although (|12L 82) may be reached, we argue that it will not change SIG]\f,N by arguing that 
for all rounds between T + 1 and the time A^ receives the full SOT broadcast for transmission 
T', there will never be codeword packets occupying a higher slot than the ghost packet. More 
precisely, we will show that for all rounds between T + 1 and the time A^ receives the full SOT 
broadcast for transmission T', either Hgp = -L or Hgp = Hjn + 1, and then by Statements 
1 and 2 of Lemma 17.11 (together with the fact that A^ is honest and so we may apply Lemma 
lll.lj) . Fill Gap on (|121 82) will not be performed (see comments on that line). That Hgp = -L 
or Hgp = HiN + 1 for all of these rounds follows from the fact that Hgp will be set to _L 
at the end of T ([I5l209), after which it can only be modified on ([12166), ([12172), ([12176), 
([12178), ([12180), or ([12182). Notice that all of these set Hgp to _L or Hin + 1 and that 
HiN + 1 cannot change for all rounds between T + 1 and the time A^ receives the full SOT 
broadcast for transmission T' by Lemma 111.81 

It remains to consider lines ([111128), ([I4ll33), ([111141), ([111146), and ([111134); the 
first four clear the signature buffers, and the last clears the broadcast buffer. So it remains 
to argue that if any of these lines are reached, either the broadcast buffer is storing all of 
the information that the signature buffers held at the end of T, or (A^, T,T') cannot appear 
as part of the SOT broadcast of transmission T'. Line (|141 128) is clearly covered by the 
latter case, since if a parcel of this form is received in some transmission T G [T + 1..T'], then 
(A'', T) is not on the sender's blacklist as of T > T, and hence (A^, T) will never be able to be 
re-added to the blacklist after this point (see (|151 188)). Similar reasoning shows that line 
(I14L 146) is covered by one of these two cases. In particular, if A^ reaches line (|141 146) in some 
transmission T G [T-|- 1..T'], then either A^ will add the information in its signature buffers into 
its broadcast buffers as on (|141 142-145) before reaching (|141 146). or else A^ was not on the 
blacklist as of T, and hence it is impossible for {N, T, T') to be a part of the SOT broadcast for 
transmission T'. Now suppose A^ reaches (|141 133-134) in some round of a transmission T > T 
indicating that a node A^ is to be eliminated. In order to reach (|14L 133-134) in transmission 
T, A^ must not have known that A'" was to be eliminated before that point (|141 131). and since 
A^ received the complete 50 T broadcast of transmission T (by Lemma fl 1 .81 together with the 
hypotheses that A^ is honest and transferred a packet in T), A^ must have been eliminated 
in some transmission T > T. In particular, if T = T, then (A^, T) can never be added to the 
blacklist (since (I14L 188) cannot be reached in transmission T if Eliminate Node is reached in 
that transmission); while if T > T, then (A^, T) will be cleared from the blacklist as on (I14L 171) 
(if it was on the blacklist), and as already remarked, (A^, T) can never again appear on the 
blacklist after this. 

Now suppose (|141 141) is reached in some transmission T > T and the signature buffers 
are cleared on this line. Now before line (I14L 141) was reached, by induction, one of the three 
statements (a), (b), or (c) was true. If (b) or (c) was true, then changes made on (|141 141) will 
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not affect the fact that (b) or (c) will remain true. Therefore, assume that we are in case (a) 
before reaching (I14L 141). i.e. that when (I14L 141) is reached in transmission T, A^'s signature 
buffers contain the information that they had at the end of T. Since (|14L 141) was reached, 
it must have been that (A'^, T,T) was received on (I14L 137) as part of the SOT broadcast for 
transmission T, for some T. We first argue T > T. To see this, since is honest, it will not 
transfer any packets in T if it is on its own version of the blacklist ( (1111 31-33) and (|111 35- 
37)). Since we know that did transfer packets in transmission T (by hypothesis), and also 
N received the full 50 T broadcast of that same transmission (Lemma lll.Sp . either A^ was not 
on the blacklist as of the start of transmission T, or A^ received information as on (I14L 147) 
indicating A^ could be removed from the blacklist. Both of these cases imply that by the end 
of T, (A^, T) can never be on the blacklist for any T < T. Thus, T > T, as claimed. Since we 
are assuming case (a), if T = T, then (I14L 141) will not be satisfied. On the other hand, if 
T > T, then A^ has appeared on the blacklist for some transmission after T, and then Lemma 
111.61 guarantees that (A^, T) is not on the blacklist as of T > T, which as noted above implies 
(A^, T, T') cannot be part of the S'OT broadcast of transmission T'. 

3. For Statement (a), we track all the times parcels are removed from A^'s broadcast buffer BB, 
and ensure that if ever A^ removes a status report parcel belonging to A^ for some transmission 
T, then (A^,T) is no longer on the sender's blacklist. If A^ = 5, notice the only place that 
information concerning other nodes' status report parcels is removed from the sender's data 
buffer is (I15L 171). and at this point A^ is not on the blacklist since the blacklist is cleared on 
this same line. 

If A^ / 5, changes to BB occur only on lines ((11134), ((141139), ((11149), ((T1142- 
145), and (|141 154). The former three lines remove things from BB, while the latter lines 
add things to BB. In terms of statement (a), we must ensure whenever one of the former 
three lines is reached, there will never be a status report parcel from A^ and corresponding 
to transmission T that is removed from BB if (A^, T) is on the blacklist. Looking first at line 
(I14L 134). suppose that A^ reaches line (I14L 134) in some transmission T > T. If (A'', T, T) was 
not a part of the SOT broadcast of transmission T, then there is nothing to show (since A^ is 
not on the blacklist as of the outset of T). So suppose that (A^,T,T) was a part of the SOT 
broadcast of transmission T. Since reaching line (|141 134) requires that A^ has newly learned 
that a node has been added to EN (I14L 131). let A^' denote this node, and let T' denote the 
round that A^' was eliminated from the network as on (I15L 170). First note that necessarily 
T' < T. After all, the blacklist will be cleared on line (|15L 171) of round T', and hence if (A^, T) 
is still on the blacklist as of the outset off, it must have been added afterwards. We now argue 
that because T' < T, the priority rules of transferring broadcast information will dictate that 
all honest nodes will necessarily learn A^' has been eliminated before they learn that (A^, T) is 
on the blacklist. From this, we will conclude that when A^ reaches (I14L 134) in transmission 
T and learns that A^' should be eliminated, that A'^ has not yet learned that (A^, T) is on the 
blacklist, and hence A^'s broadcast buffer will not be storing any of A^'s status report parcels 
for T ((11152). 

It remains to show that any honest node A ^ G will learn that A^' has been eliminated 
before they learn (A^, T) is on the blacklist. So fix an honest node A ^ G. Suppose A first 
learns (A^, T) is on the blacklist via a parcel of the form (A^, T, X) that it received as on (I14L 137) 
of transmission X. Clearly, X > T, since (A^, T) can only be put on the blacklist at the very 
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end of transmission T. Therefore, since T' < T < X, we have that {N', X) will be a part of the 
50 T broadcast for transmission X, indicating that N' has been eliminated (I15L 200). Since 
A is honest, it will therefore receive (A^',X) before it receives (A^,T,X) (see priority rules for 
receiving broadcast parcels, (|131 110) and (I13L 115)) 

We next consider when status report parcels are removed from BB as on (|14L 139). In this 
case, has received a S'OT broadcast parcel of form (A^,T, T') (I14L 137). and is removing 
from BB all of A^'s status report parcels corresponding to transmissions other than T. First 
note that Lemma 111.61 guarantees that N is on at most one blacklist at any time. Since 
received a S'OT parcel of the form (A^, T, T') during transmission T', it must be that (A^, T) was 
on the sender's blacklist at the outset of T', and since nothing can be added to the blacklist 
until the very end of a transmission (I15L 188). only (N,T) can be on the sender's blacklist at 
the outset of T'. This case is now settled, as we have shown that A'' does not remove any of the 
status report parcels from A^ corresponding to T on (I14L 139). and this is the only transmission 
for which A^ can be on the blacklist (at least through T'). 

To complete Statement (a), it remains to consider line (|141 149). But this is immediate, 
as if the sender at any time removes (A^, T) from the blacklist, then it can never again be 
re-added (since nodes are added to the blacklist at the very end of a transmission (|151 188). 
they are not removed as on (|141 166) or (|15L 171) until at least the next transmission, at which 
point the same [node, transmission) pair (A^,T) can never again be added to the blacklist as 
on (|151 188) since T has already passed). Therefore, when A^ reaches (|141 149). if the items 
deleted from BB correspond to A^, then A^ must have received a broadcast parcel of form 
(A^,0,T) as on (I14L 147). indicating that A^ was no longer on the blacklist. Consequently, 
the status parcels deleted will never again be needed since (A^, T) can never again be on the 
blacklist. 

Part (a) of Statement 3 of the lemma (now proven) states that no status report parcel still 
needed by the sender will ever be deleted from a node's broadcast buffer. Part (b) states that 
a node's broadcast buffer will not hold extraneous status report parcels, i.e. status reports 
corresponding to multiple transmissions for the same node. This is immediate, since whenever 
a node A^ learns a node (A^,T') is on the blacklist as on (I14L 137). then A^ will immediately 
delete all of its status report parcels from A^ corresponding to transmissions other than T' 
(I14L 139). The fact that the stored parcels have the correct information (i.e. that they address 
the appropriate reason for failure as on (|141 142-145)) follows from the fact that N will only 
initially store a status report parcel if it contains the correct information (|141 153). 

4. There are three lines on which the broadcast parcels of the kind relevant to Statement 4) are 
removed from A^'s broadcast buffer: (I14L 134). (|141 139). and (I14L 149). We consider each of 
these three lines. Suppose first that the parcel {B, N, T) is removed from A^'s broadcast buffer 
as on line (I14L 134) of some transmission T'. In particular, A^ learns for the first time in the 
SOT broadcast of transmission T' that some node A^ has been eliminated. Let T denote the 
transmission that the sender eliminated this node (as on (|151 169-177)). If T > T, then (A^,T) 
will be cleared from the blacklist on line (I15L 171) of T, and hence when (BjN,!) is removed 
from A'^'s broadcast buffer in transmission T' > T, (A^,T) will no longer be on the blacklist, as 
required. Therefore, assume T < T (equality here is impossible since lines (1151 169-177) and 
(I15L 188) can never both be reached in a single transmission, see e.g. (I15L 177)). Let X denote 
the transmission in which A^ first learned that (A^, T) was on the blacklist, i.e. A^ received a 
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parcel of the form (A^, T, X) on (I14L 137) of transmission X. Clearly, X > T, since (A^, T) can only 
be added to the blacklist at the end of T (I15L 188). Also, X < T', since by hypothesis a parcel of 
the form {B, N,"!) is removed from A'^'s broadcast buffer on line (|14L 134) of T', and this parcel 
can only have been added to A'"'s broadcast buffer in the first place if A^ already knew that 
(A^,T) was blacklisted (|14L 151). Lastly, X > T', since T < T implies that A^ was eliminated 
before {N,T) was added to the blacklist, and therefore by the priorities of sending/receiving 
broadcast parcels ( (|13L 110) and (|131 115)). we have that an honest A^ will learn that A^ has 
been eliminated before it will learn that (A^, T) is on the blacklist. Combining these inequalities 
shows that X > T' and X < T', so X = T'. But this implies that when (I14L 134) is reached in 
T', A'^ does not yet know that (A^,T) is on the blacklist, and consequently the parcel {B,N,'I) 
cannot yet be stored in A'^'s broadcast buffer, which contradicts the fact that it was removed 
on (I14L 134) of T'. Therefore, whenever (I14L 134) is reached, either (A^, T) will no longer be 
on the blacklist, or there will be no parcels of the form {B,N,T) that are removed. 

Suppose now that the parcel {B, N, T) is removed from A^'s broadcast buffer as on line 
(I14L 139) or (|141 149) of some transmission T'. In either case, by looking at the comments on 
these lines together with Lemma 111.61 (A^, T) has already been removed from the blacklist if 
a parcel of the form (B, N, T) is removed on either of these lines. ■ 

Lemma 11.17. If A,B £ G are honest (not corrupt), in any transmission T for which both A and 
B have received the full SOT broadcast: 

1. Between the time B accepts a packet from A on line 1121 77 ) through the time A gets confir- 
mation of receipt (see Definition \7.6\} for it as on ( 1121 50 ). we have: 

. SIG^[1]a,b = 1 + SIG''[1]a,iE 
. SIG^[p]a,b = 1 + SIG^[p]a,b^^ 

• SIG^[2]a,b = M + 5/G^[3]a,b, where M is the value of Hpp on ^2^60) (according to 
A's view) in the same round in which ( 1121 77) was reached by B 

• SIG^[3]a,b = m + SIG^[2]a,b, where m is the value of Hgp on ( 1121 75 ) (according to 
B's view) in the same round in which 1121 77) was reached by B 

2. At all other times, we have that SIG^[1]a,b = SIG^[1]a,b, SIG^[2\a,b = SIG^[3]a,b, 
SIG^[i\A,B = SIG^^]a,b, and SIG^\p\a,b = SIG^\p]a,b for each packet p that is part of 
the current codeword. 

Proof. The structure of the proof will be as follows. We begin by observing all signature buffers are 
initially empty (|101 48) and (I10L 54). and that for any transmission T, both SIG^ and SIG^ are 
cleared before any packets are transferred (Lemma lll.Sj) . We will then focus on a single transmission 
for which A and B have both received the full 50 T broadcast, and prove that all changes made to 
SIG^ and SIG^ during this transmission (after the buffers are cleared upon receipt of the SOT 
broadcast) respect the relationships in the lemma. Since the only changes occur on lines (1121 48-49) 
and (|12L 74-75). it will be enough to consider only these 4 lines. Furthermore, if lines (1121 48-49) 
were reached x times by A in the transmission, and lines (|12l 74-75) were reached y times by B, 
then: 

""^If the packet accepted corresponds to an old codeword, then SIG^[1\a,b = SIG^[1\a,b and SIG^[p\a,b = 
S/G"*[1]a,s = ±. 
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(a) Either y = xory = x + l, 

(b) Neither set of hnes can be reached twice consecutively (without the other set being reached 
in between) 

(c) Lines (I12L 74-75) are necessarily reached before lines (I12L 48-49) (i.e. in any transmission, 
necessarily y will change from zero to 1 before x does). 

Notice that the the top statement follows from the second two statements, so we will only prove 
them below. 

We first prove the three statements above. We first define x more precisely: x begins each 
transmission set to zero, and increments by one every time line 50 is reached (just after A^s signature 
buffers are updated on lines (|12L 48-49)). Also, define y to begin each transmission equal to zero, 
and to increment by one when line (|12L 74) is reached (just before B^s signature buffers are updated 
on lines (1121 74-75)). Statement (c) is immediate, since RR begins every round equal to —1 (lines 
dlOlSO) and ([151209)), and can only be changed to a higher index on ([T2l78). Therefore, ([12146) 
can never be satisfied before (|121 78) is reached, which implies (I12L 48) is never reached before 
(|121 74) is. We now prove Statement (b). Suppose lines (|12l 48-49) are reached in some round t. 
Notice since we are in round t when this happens, and because RR can never have a higher index 
than the current round index, and the most recent round RR could have been set is the previous 
round, we have that -B's value for RR (and the one A is using on the comparison on (I12L 45-46)) 
is at most t — 1. Also, Hpp and FR will be set to _L on (|121 51) of t. If FR ever changes to a 
non-_L value after this, it can only happen on (|121 56). and so the value it takes must be at least t. 
Therefore, if at any time after t we have that FR ^ _L, then if RR has not changed since t — 1, 
then (I12L 46) can never pass, since RR < t — 1 < t < FR. Consequently, (|121 78) must be reached 
before (|12l 48-49) can be reached again after round t, and hence so must (I12L 74-75). This shows 
that (|12l 48-49) can never be reached twice, without (I12L 74-75) being reached in between. 

Conversely, suppose lines (I12L 74-75) are reached in some round t. Notice since we are in round 
t when this happens, and because FR can never have a higher index than the current round index, 
we have that A^s value for FR (and the one B is using on the comparison on (I12L 73)) is at most t. 
Also, RR will be set to t on (|121 78) of round t, and RR cannot change again until (at some later 
round) (|12L 73) is satisfied again (or the end of the transmission, in which case their is nothing to 
show). If line (|121 56) is NOT reached after (|12l 74-75) of round t, then FR can never increase to a 
larger round index, so FR will remain at most t. Consequently, line (|12L 73) can never pass, since if 
B receives the communication from A on line (|121 62). then by the above comments RR > t > FR. 
Consequently, (|121 56) must be reached before (|121 73) can be reached again after round t. However, 
by Statement 3 of Lemma ITTTI (I12L 56) cannot be reached until A receives confirmation of receipt 
from B (see Definition 17. 6p . i.e. (|121 56) can be reached after (I12L 74-75) of round t only if lines 
([12148-49) are reached. 

We now prove the lemma by using an inductive argument on the following claim: 
Claim. Every time line ft 21 74) is reached (and y is incremented), we have that equalities of 
Statement 2 of the lemma are true, and between this time and the time line 1121 48 ) is reached (or 
the end of the transmission, whichever comes first), we have that the equalities of the first statement 
of the lemma are true. 

To prove the base case, notice that before lines (1121 74-75) are reached for the first time, but after 
both nodes have received the transmission's 50 T broadcast, all entries to both signature buffers are 
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_L, and so the induction hypothesis is true. Now consider any time in the transmission for which 
y is incremented by one in some round t (i.e. hue (I12L 74) is reached). Since neither x nor y can 
change between hues (|11L 20) and (|111 22). by the induction hypothesis we have that the equahties 
of the second statement of the lemma are true when A sends the communication as on (I12L 60) 
of round t. Since A has actually sent {SIG^[\] + 1, SIG^\p\ + \, SIG^[i\ + Hpp), and these are 
the quantities that B stores on lines (1121 74-75). we have that the first statement of the lemma 
will be true after leaving line (I12L 75) (and in particular the claim remains true). More specifically, 
letting M denote the value of Hpp (respectively letting m denote the value of Hop) when (I12L 60) 
(respectively (|121 74)) is reached in round t, we will have that immediately after leaving (|121 75): 

1. SIG^[1]a,b = 1 + SIG^[1]a,b 

2. SIG^[p]a,b = 1 + SIG^[p]a,b 

3. SIG^[2]a,b = M + SIG^[3]a,b 

4. SIG^[3]a,b = m + SIG^[2]a,b 

as required by Statement 1 of the Lemma. By Statement (b) above, either the signature buffers along 
E{A, B) do not change through the end of the transmission, or the next change necessarily occurs 
as on (1121 48-49). In the former case, the Claim certainly remains true. In the latter case, let t' 
denote the time that (I12L 48) is next reached. Notice that t' > t, as Statement (b) above guarantees 
(|121 48) is reached after (|12L 74). and by examining the pseudo-code, this cannot happen until at 
least the next round after t. In particular, the values received on (I11L 07) of round t' necessarily 
reflect the most recent values of SIG^ (i.e. S's signature buffers have already been updated as on 
(|12l 74-75) when B sends A the communication on (|12l ll)). Consequently, A will change SIG^[1]^ 
SIG^[2l and SIG^[p] to the values B is storing in SIG^[l], 5IG^[3], and respectively. 
Therefore, the claim (and hence the lemma) will be true provided we can show that when A updates 
as on (|12L 49). that the new value for SIG^[3\ equals the value stored in 5/G^[2]. Since 
before (|121 49) is reached, we have by the induction hypothesis that SIG^\1]a,b = M^+SIG"^^]a,b, 
it is enough to show that when 5/G^[3] is updated on (|12L 49). that the value of Hpp there equals 
M . We argue that this by showing Hpp will not change from line (|12L 60) of round t (when M was 
set to Hpp) through line (|121 49) of round t'. To see this, notice that the only possible places Hpp 
can change during a transmission are lines (I12L 51). (|121 53). and (|121 56). Clearly, (|121 51) cannot 
be reached between these times, since (I12L 49) is not reached during these times. Also, Statement 3 
of Lemma 17.71 implies that (|121 56) cannot be reached between these times either. Finally, (I12L 53) 
cannot be reached, since RR will be set to t on (I12L 78) of round t, and by statement (b), (I12L 78) 
cannot be reached again until after (|121 49) is reached in round t', and hence RR will be equal to 
t from (I12L 78) of round t through (I12L 49) of round t'. Also, FR will not change between these 
times (also by Statement 3 of Lemma YHh . and since the only non-_L value FR is ever set to is the 
current round as on (|121 56). we have that FR < t. Putting these facts together, we have that 
for all times between line (|121 60) of round t through line (I12L 49) of round t', either A does not 
receive RR (in which case RR = _L when (|12L 52) is reached) or A receives RR, which as noted 
obeys RR = t > FR. In either case, (I12L 52) will fail, and (I12L 53) cannot be reached. ■ 

Lemma 11.18. For any transmission T, recall that Vj denotes the list of nodes that participated 
in that transmission, and it is set at the end of each transmission on ft5L 187). For any honest (not 
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corrupt) node A E G, during any transmission T, A will not exchange any codeword packets with 
any node that is not put on Vj at the end of the transmission. 

Proof. Restating the lemma more precisely, for any node N that is NOT put on V-y as on (I15L 187) 
and for any honest node A ^ G, then along (directed) edge E{A., N), A will never reach line (|111 60). 
and along (directed) edge E{N,A), A will never reach lines (|12l 67-82). Fix a transmission T in 
which (I12L 187) is reached (i.e. a node is not eliminated as on (I15L 169-177) of T), let N ^ Vj he 
any node not put on on (I15L 187) of T, and let A € G be an honest node. Since N ^ Vj, we have 
that either G EN oi N € BL when (I15L 187) is reached. Since no nodes can be added to EN or 
BL from the outset of T through line (I15L 187) of T, we must have that G EN or N G BL as of 
either line (I15L 188) or (|151 170) of the previous transmission. Therefore, either (A^, T) or (A^, T',T) 
is added to the S'OT broadcast of transmission T (on (|151 176) or (|151 200) of transmission T — 1), 
indicating A^ is an eliminated/blacklisted node. If A has not received the full Start of Transmission 
(SOT) broadcast for T yet, then the lemma is true by Lemma [11.81 If on the other hand A has 
received the full SOT broadcast, then in particular A has received the parcel indicating that A^ is 
either eliminated or blacklisted. Thus, by lines ^59), ([11131-33), ^63) and ([11135-37), A 
will not transfer any packets with A^. ■ 

Lemma 11.19. The receiver's end of transmission broadcast takes at most n rounds to reach the 
sender. In other words, the sender will have always received the end of transmission broadcast by 
the time he enters the Prepare Start of Transmission Broadcast segment on 1111 29 ). 

Proof. By the conforming assumption, for every round t of every transmission there is a path Pt 
between the sender and receiver consisting of edges that are always up and nodes that are not 
corrupt. We consider the final n rounds of any transmission, and argue that for each round, either 
the sender already knows the end of transmission parcel O, or there is a new honest node N ^ G 
that learns O for the first time. Since the latter case can happen at most n — 1 times (the receiver 
already knows O when there are n rounds remaining, see (I11L 28) and (1151 178-179)). it must be 
that the sender has learned G by the end of the transmission. Therefore, let AD — n < t < AD be 
one of the last n rounds of some transmission. If the sender already knows G, then we are done. 
Otherwise, let Pt = NqNi . . . Nl (here Nq = S and A''^, = R) denote the active honest path for 
round t that connects the sender and receiver. Since 5 does not know G but R does, there exists 
some index < i < L such that Ni does not know G but A^j+i does know G. Since edge E{Ni, Ni^i) 
is active and the nodes at both ends are honest (by choice of Pt), node A'j+i will send Ni a broadcast 
parcel on (I11L 15). Looking at the manner in which broadcast parcels are chosen (|131 115). it must 
be that A'j+i will send Q to Ni in round t, and hence A'^j will learn G for the first time, which was 
to be showed. ■ 

Lemma 11.20. // the receiver has received at least D — 6n^ distinct packets corresponding to the 
current codeword, he can decode the codeword (except with negligible probability of failure). 

Proof. Fact 1' guarantees that if the receiver obtains D — 6n^ distinct packets corresponding to 
a codeword, then he can decode. Since all codeword packets are signed by the sender to prevent 
modifying them, the security of the signature scheme guarantees that any properly signed codeword 
packet the receiver obtains will be legitimate (except with negligible probability of failure). ■ 

Lemma 11.21. For every transmission T; S,R€z Vj. 
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Proof. The participating list Vj is set at the end of every transmission on Hne (|15L 187). By looking 
at the code there, we must show that S, R ^ EN U BL at the end of any transmission. That an 
honest node can never be identified as corrupt and eliminated is the content of the proof of Theorem 
\8.1\ so S,R ^ EN. Since S is never put on the blacklist (1151 188). it remains to show R ^ BL 
when (|151 187) is reached. Since nodes are removed from the blacklist on line (I14L 166) and not put 
on it again until (I15L 188). it is enough to show that if R is ever placed on the blacklist at the end 
of some transmission T — 1, then it will be removed as on (I14L 166) of transmission T. If R is ever 
placed on the blacklist, we argue that: 1) R will learn what status report parcels the sender requires 
of it after at most 2n^ rounds; and 2) S will receive all of these parcels by at most 4n^ rounds 
later. Therefore, R will necessarily be removed from the blacklist by round 4n^ + 2n^ < AD (since 
D > 6n^), as required. To prove 1), first note that all honest nodes remove the receiver's end of 
transmission parcel for T — 1 at the very end of T — 1 (I15L 203). Therefore, no honest node will have 
any End of Transmission Parcel in its broadcast buffer at any point during T until one is created 
for the current transmission on (1151 178-179). Therefore, for the first rounds, the sender's SOT 
broadcast will have top priority in terms of sending/receiving broadcast parcels (|13L 115). Since S 
and R are connected by an active honest path at each round, we follow the proof as in Lemma fl 1.191 
to argue that for every round between the outset of T and round n^, either R has learned the full 
50 T broadcast, or there is an honest node that is learning a new 50 T broadcast parcel for the first 
time. Since there are (at most) n nodes, and the S'OT broadcast has at most 2n parcels (see proof 
of Lemma [11.21 and Statement 2 of the Broadcast Buffer therein), it takes at most 2n? rounds for 
R to receive the full S'OT broadcast, and hence to learn it has been blacklisted. This proves 1). 

Upon receipt of this information, R adds the necessary information (i.e. its status report) 
to its broadcast buffer (|14L 137-145). Looking at the proof of Theorem 110.91 and in particular 
Claim 2 within the proof, edges along the active honest path can take at most 4n^ < AD rounds 
to communicate across their edges the broadcast information of priorities 1-6 on lines (|131 115). 
and since the receiver is connected to the sender every round via some active honest path (by the 
conforming assumption), its requested status report information will necessarily reach the sender 
within 4n^ rounds, proving 2). ■ 

Lemma 11.22. For any transmission T, if Vj = {S,R}, then the transmission was necessarily 
successful. 

Proof. Vj is set on line (|151 187). Since the only place the sender adds nodes to the blacklist 
is on (I15L 188). which happens at the very end of each transmission, and because the hypothesis 
states that every non-eliminated node except for S and R is on the blacklist when line (I15L 187) of 
transmission T is reached, it must be the case that transmission T began with every non-eliminated 
node on the blacklist, with the possible exception of the receiver (and the sender who is never 
blacklisted). Since all internal nodes are still blacklisted by the end of the transmission, the sender 
will never transfer any packets to any node other than R during transmission T (line (1121 59) will 
always fail for any other node, see (|llL 31-33)). Theorem 1 1 . 91 indicates there are at most 4n^ rounds 
that are wasted, and since the only edge the sender can ever use to transfer codeword packets during 
T is E{S,R), the conforming assumption implies edge E{S,R) is active every round of T. We may 
therefore view the graph as reduced to a single edge connecting S and R (see Lemma ril.lSjl . where 
there are at least 4D — 4n^ > 3D (non-wasted) rounds per transmission. Since both S and R are 
honest, correctness is guaranteed as in the edge-scheduling protocol by Lemma fl 1.1 1 In particular, 
the transmission will necessarily be successful. ■ 
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Lemma 11.23. No honest node will accept more than one distinct parcel (per node N per trans- 
mission) indicating that N should he removed from the blacklist. 

Proof. Line (|131 110) guarantees that any node A will only accept the parcel if it has already 
received the sender's start of transmission broadcast corresponding to the current transmission. In 
particular, this means that A has received an updated blacklist (and a list of eliminated nodes) 
before it accepts any removals from the blacklist. Therefore, in some transmission T, if A ever does 
accept the information that a node should be removed from the blacklist, then this information 
will not become out-dated until (if) is added to the blacklist again, which can happen at the 
earliest at the very end of transmission (|151 188). Therefore, after receiving the information for 
the first time that N should be removed, the comments on line (I14L 123) will guarantee A will not 
accept additional blacklist information regarding A'^ until the following transmission, proving the 
lemma. ■ 

Lemma 11.24. For any node N €z G, after receiving the complete SOT broadcast, an honest node 
N will transmit along each edge at most once per transmission the fact that it knows N 's complete 
status report. 

Proof. Each parcel stored in A'^'s broadcast buffer BB is accompanied by a list of which edges the 
parcel has been successfully transmitted across (see comments on line (|141 123)). Therefore, as long 
as the parcel is not deleted from the broadcast buffer, line (I13L 115) guarantees that each parcel 
of broadcast information will only pass along each edge once, as required. Therefore, it remains 
to prove the lemma in the case that the relevant broadcast parcel is deleted at some point in a 
transmission. Fix a transmission T and an arbitrary N € G. Since broadcast parcels of the relevant 
type (i.e. that A'^ has A^'s complete status report) are only removed on (I14L 139) and (I14L 149). we 
need only consider the case that (|141 149) is reached in transmission T (the former line can only 
be reached as part of the 50 T broadcast, and therefore lies outside the hypotheses of the lemma). 
In particular, we will show that if (I14L 149) deletes from A^'s broadcast buffer the parcel indicating 
that A^ knows A'^'s complete status report, then N will never again add a parcel of this form to 
its broadcast buffer (as on (I14L 155)) for the remainder of T. But this is immediate, since if A^ 
removes this parcel from BB on (I14L 149) of T, then A^ must have been removed from the blacklist 
(see (I14L 147)). and since A^ cannot be re-added to the blacklist until the end of T (|151 188). line 
(|141 152) (of A^'s code, with the A^ that appears there equal to the A^ used in the present notation) 
cannot be satisfied for the remainder of T, and hence (|141 155) cannot be reached. This proves that 
once the parcel is deleted, it cannot be later added in the same transmission, proving the lemma. ■ 

12 Conclusion and Open Problems 

In this paper, we have described a protocol that is secure simultaneously against conforming 
node-controlling and edge-scheduling adversaries. Our results are of a theoretical nature, with 
rigorous proofs of correctness and guarantees of performance. Surprisingly, our protocol shows 
that the additional protection against the node-controlling adversary, on top of protection against 
the edge-scheduling adversary, can be achieved without any additional asymptotic cost in terms of 
throughput. 

While our results do provide a significant step in the search for protocols that work in a dynamic 
setting (edge-failures controlled by the edge-scheduling adversary) where some of the nodes are 
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susceptible to corruption (by a node-controlling adversary), there remain important open questions. 
The original Slide protocoo requires each internal node to have buffers of size 0(n^ log n), while 
ours requires O(n^logn), though this can be slightly improved with additional assumption^. In 
practice, the extra factor of may make our protocol infeasible for implementation, even for 
overlay networks. While the need for signatures inherently force an increase in memory per node 
in our protocol verses the original Slide protocol, this is not what contributes to the extra O(n^) 
factor. Rather, the only reason we need the extra memory is to handle the third kind of malicious 
behavior, which roughly corresponds to the mixed adversarial strategy of a corrupt node replacing 
a valid packet with an old packet that the node has duplicated. Recall that in order to detect this, 
for every packet a node sees and for every neighbor, a node must keep a (signed) record of how 
many times this packet has traversed the adjacent edge (the O(n^) packets per codeword and 0{n) 
neighbors per node yield the O(n^) bound on memory). Therefore, one open problem is finding a 
less memory-intensive way to handle this type of adversarial behavior. 

Our model also makes additional assumptions that would be interesting to relax. In particular, it 
remains an open problem to find a protocol that provides efficient routing against a node-controlling 
and edge-scheduling adversary in a network that is fully asynchronous (without the use of timing 
assumptions, which can be used to replace full synchrony in our solution) and/or does not restrict 
the adversaries to be conforming. As mentioned in the Introduction, if the adversary is not conform- 
ing, then he can simply permanently disconnect the sender and receiver, disallowing any possible 
progress. Therefore, results in this direction would have to first define some notion of connectedness 
between sender and receiver, and then state throughput efficiency results in terms of this definition. 
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